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Attachment 1 



^ #% '\ UNITED STATES ENVIRONMENTAL PROTECTION AGENCY 

I 52S& ^ WASHINGTON, D.C. 20460 



December 15,2005 
FROM: Marcus Peacockl^ 



.^ 



DEPim' ADMINISTRATOR 



TO: Dr. George Gray 

Assistant Administrator for the Office of Research and Development 

Bill Wehrum 

Acting Assistant Administrator for the Office of Air and Radiation 

RE: National Ambient Air Quality Standards 

An essential component of EPA's mission is to ensure that the best available 
science guide and inform agency decision making. This principle has been and 
continues to be a major focus of the agency. Under the heading "Achieving Results," the 
2003-2008 EPA Strategic Plan notes that, "EPA's approach to conducting and using 
science in service to the Agency's mission will ensure that Agency policies, decisions, 
and other activities reflect high-quality scientific information relevant to current and 
future environmental issues." EPA must follow guidelines and procedures so that "our 
work meets the highest standards of scientific excellence." 

Using best available science to accelerate enviroiunental progress and protect 
public health is a top priority for the Administrator. To help achieve this goal, the 
Administrator has asked me to examine whether the process for setting National Ambient 
Air Quality Standards (NAAQS) can be strengthened. The current NAAQS process has 
been in place for over 20 years, with some aspects required by law, and therefore not 
amenable to changes except through new legislation. Other important aspects of the 
NAAQS process^ however, are discretionary— t he a gency has established practices that 
set parameters for how science supports decision making. The Administrator is 
interested in determining whether those practices reflect the most rigorous, up-to-date, 
and unbiased scientific standards and methods. 

To fulfill the Administrator's directive, I am asking you to establish and co-chair 
an agency working group to determine whether (and if so, how) the agency utiUzes the 
best available science to set the NAAQS. hi its examination, the working group should 
conduct a top-to-bottom review of the NAAQS process. 

The working group should consider four reports (1998, 1999, 2000, and 2004) 
conducted by the "Committee on Research Priorities for Airborne Particulate Matter," 
which was created by the National Academy of Sciences (NAS), as well as "Science and 
Judgment in Risk Assessment," in which the NAS offered specific recommendations on 
how EPA can improve its risk assessments. 
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In addition, the working group should focus on the nexus between scientific 
analysis and standard setting, including the degree to which we are successful in 

management strategies and policy judgments. It should also examine whether the process 
for compiling the Criteria Document allows for consideration of the most relevant, 
objective, and up-to-date scientific data. The working group should address and propose 
ways to ensure broad participation of the scientific community in the NAAQS process. 
Finally, the working group should consider whether and how these objectives can be 
satisfied within the statutory 5 year review period. 

It is imperative that the NAAQS process adheres to the highest scientific 
standards so we can continue to make environmental progress and protect public health. 
The working group is charged with issuing specific recommendations to me no later than 
April 3, 2006. I look forward to working with you and the agency's top scientific experts 
on this very important subject. 
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Attachment 2 

Key Questions for the Review of the Process for Setting NAAQS 

Timeliness of the NAAQS review process 

o What are your views on the timeliness and efficiency of the current process for 

both EPA's and CASAC's reviews of the air quality criteria and the NAAQS, in 
terms of the time that is spent between the start of the review and the publication 
of the Agency's proposed decisions on the standards? 

o Can you identify structural changes to the process and/or key documents (e.g., the 

Criteria Document, Staff Paper, Risk Assessment) or changes in the Agency's 
management of the process that could shorten this timeframe while preserving an 
appropriately comprehensive, transparent and policy-relevant review and 
allowing adequate opportunities for CASAC review and advice and for public 
comment on these documents? 

Consideration of the most recent available science 

o To enhance the Agency's ability to take the best and most recent available science 

into account in making decisions on the standards, can you suggest changes in the 
process and/or key documents that could shorten the time between the 
presumptive cutoff date for scientific studies evaluated in the review and reaching 
proposed decisions on the standards, or that could otherwise facilitate appropriate 
consideration of more recent studies? 

Distinctions between science and policy judgments 

o Recognizing that decisions on the standards, while based on the available science, 

also require policy judgments by the Administrator, what are your views on how 
clearly scientific information, conclusions, and advice are distinguished from 
policy judgments and policy recommendations on the standards throughout the 
review process? 

o Can you suggest changes in the process and/or changes to the format and contents 

of key documents that would help to make these distinctions clearer? 

Identifying, characterizing, quantifying, and communicating uncertainties in scientific 
information 

o Recognizing the importance of characterizing and clearly communicating the 

uncertainties in the science and quantifying uncertainties in exposure and risk 
estimates as explicitly as possible, what are your views on any changes in the 
process and/or changes to the format and content of key documents that might 
facilitate a more complete, quantitative, and policy-relevant characterization of 
uncertainties? 
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^^^^°'^^^e-. 



•* 



UNITED STATES ENVIRONMENTAL PROTECTION AGENCY 



I ^1/^ J WASHINGTON D.C. 20460 



OFFICE OF THE ADMINISTRATOR 
SCIENCE ADVISORY BOARD 

March 16, 2006 

MEMORANDUM 

SUBJECT: Clean Air Scientific Advisory Committee Members' Comments on the Agency's 
Process for Establishing National Ambient Air Quality Standards 

FROM: VanessaT.Vu, Staff Director /signed/ 

EPA Science Advisory Board 

TO: William Wehrum 

Assistant Administrator for Air and Radiation 

George Gray 

Assistant Administrator for Research and Development 

In response to your memorandum to me dated February 17, 2006, 1 sent invitations to 
current and former members of the Clean Air Scientific Advisory Committee (CASAC), asking 
for their individual comments regarding EPA's process by which the National Ambient Air 
Quality Standards (NAAQS) for "criteria" air pollutants are established. As of March 16, we 
have received seven (7) sets of comments, which are attached for your consideration. 

Thank you for requesting CASAC s input into this important Agency review. 



Attachment 
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Attachment : CAS AC Members' Individual Comments on EPA's NAAQS Process 

Member Page# 

Dr. Ellis Cowling A-2 

Dr. Bernard D. Goldstein A- 11 

Dr. Rogene F. Henderson A-13 

Dr. Philip K. Hopke A-17 

Dr. Morton Lippmann A-20 

Dr. Joe Mauderly A-23 

Dr. George T. Wolff A-26 
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Dr. Ellis Cowling 

Dr. Ellis Cowling 
North Carolina State University 

March 15, 2006 

Comments on the Experience and Value of CASAC's Advice and Counsel in Assisting EPA in 
Establishing National Ambient Air Quality Standards for Criteria Pollutants 

The current call for a "top-to bottom" review of the processes used by EPA and its 
Congressionally-authorized Clean Air Science Advisory Committee (CASAC) is the latest in a 
long series of reviews of NAAQS-development processes during the past three and one-half 
decades since the creation of EPA in 1970. The reviews of which I am aware (and which I have 
reviewed once again in recent weeks) include evaluative reports on CASAC's performance in 
service to EPA in 1979, 1981, 1983, and 1987 in addition to the peer-reviewed paper on this 
same subject published by Morton Lippmann in 1987. In the context of the present debate, I 
commend each of these evaluative reports for the wisdom they collectively bring to the challenge 
of designing an optimum system by which NAAQS should be developed in the future. 

One year ago now, Philip Hopke stepped down from his responsibilities as Chair of CASAC. 
In his recent written "Comments on the NAAQS Review Process," Hopke has provided a series 
of carefully considered recommendations that derive from his long and experience as an 
effective leader of CASAC. I also commend his recommendations for consideration in the 
context of the present "top-to-bottom" review of the NAAQS development processes - 
especially: 

1) His direct references to the explicit directives of the US Congress in establishing CASAC 
and defining its membership, its duties to "recommend to the Administrator any new 
ambient air quality standards," and to "advise the Administrator [about] "research efforts 
necessary to provide the required information" and "any adverse . . . effects which may 
result from various strategies for attainment and maintenance of such national ambient air 
quality standards." 

2) His firm comments on the "Distinctions between Science and Policy Judgments," and 

3) His specific recommendation that the centerpiece of every Criteria Document should be 
the "Integrative Synthesis Chapter" [and I would add, a very carefully crafted "Executive 
Summary"] for each Criteria Document and Staff Paper. Both of these major documents 
should be written with careful recognition of "the major take home messages" from each 
document. The intent being to focus CASAC's scientific reviewing attention on "the 
content that matters most" in informing final policy decisions by the Administrator of 
EPA. 

Although I am a relatively recent (2004) appointee to Membership in CASAC, my 
impression is that the scientific review processes used can be improved substantially by taking 
full advantage of the experience and insights from other organizations that produce high-quality 
science-based policy analysis products or have studied in a rigorous way the processes by which 
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high-quality assessment documents are produced. These other organizations include the 
following: 

1) The special science- and policy-focused subject-matter committees established by the 
National Research Council (NRC) within the National Academy of Sciences (NAS) — 
with detailed oversight and review of all NRC reports provided by the Academies' 
demanding Report Review Committee, 

2) The authoritative analyses of the science-and-policy assessment processes used in various 
countries of the world. Two of these very rigorous analyses of the processes and quality 
of the resulting assessment documents have been produced by Dr. William Clark and 
other leaders for the "Social Learning" and "Global Environmental Assessment" projects 
at the Kennedy School of Government at Harvard University, and 

3) The science and policy interface recommendations developed by the Oversight Review 
Board of the National Acid Precipitation Assessment Program led by Milton Russell, 
former Assistant Administrator for EPA, Chauncey Starr, former Director of Research for 
the Electric Power Research Institute (EPRI), Tom Malone, former Foreign Secretary for 
the National Academy of Sciences, John Tukey, Distinguished Professor of Statistics at 
Princeton University, and Kenneth Starr, Nobel Prize Winner in Economics. 

It was my good fortune to have the opportunity to participate in all three of these 
organizations during the past 15-20 years: 

1) As a member of several special science and policy subject matter committees dealing 
with air-quality relevant reports by the NRC, and later, as a Member for several 
additional years in the Report Review Committee of the NRC; 

2) As an Adjunct Faculty Member in both the "Social Learning" and "Global Environmental 
Assessment" projects at the Kennedy School of Government; and 

3) As the Liaison between the Office of the Director of NAPAP and the NAPAP Oversight 
Review Board led by Milton RusseU. 

From many lessons learned from these three experiences, permit me to offer the following 
generalizations regarding the value and limitations of science, and the appropriate roles that 
scientists, engineers, policy analysts, and decision makers can and should play in making 
science-based public policy decisions in general — and in particular — in the establishment of 
National Ambient Air Quality Standards and in management of air quality in our country: 

Science is the discovery of new knowledge through research. Scientific inquiry is driven 
mainly by the curiosity and enthusiasm of individuals, and groups of scientists and engineers. 
Scientific inquiry involves hypotheses, measurements, testing, and development of 
conclusions and concepts based on inductive and deductive reason. Basic research is inquiry 
aimed at understanding the physical, biological, social, and mathematical world around us. 
Applied research is inquiry aimed at discovering useful guidelines for management of air 
quality, natural resources, business enterprises, social and governmental institutions, and the 
health, educational, recreational, and other services needed by society. 

Science provides the power to understand natural phenomena, and, by virtue of that 
power, to expand the range of choices for management of nature and human institutions. 
Technology is the art of making things useful. Technological innovation provides the means 
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by which the power of science can be harnessed to drive the economic and social systems of 
society by providing new products, processes, and services that are needed by society. 

But science and technology alone cannot provide the wisdom to make wise choices. 
Wisdom derives not only from science and engineering, but also from the humanities. For 
this reason, the power of scientific and technological innovation must be balanced and 
focused for the benefit of society by reflective study of philosophy, justice, aesthetics, 
history, religious faith, and all the pain and suffering as well as the joys and satisfactions of 
individuals and groups within society. 

Assessment leading to formulation of wise public pohcy is a process by which scientific 
and technological evidence is marshaled for the purposes of predicting the outcomes of 
alternative courses of action. Assessment is not driven by individual or even collective 
curiosity. In its best form, assessment is driven by a prescribed set of policy-relevant issues 
that ideally are written down in a coherent set of policy-oriented questions for which precise 
answers, having to do with both science and societal values, are central. Assessment 
involves analyzing the quality of scientific understanding and identifying and bounding 
uncertainty so that decision makers can act with an appropriate interpretation of the benefits, 
costs, values held dear, and risks that are expected to be associated with alternative courses 
of action. 

There are four actor groups — each with its own distinctive role and responsibility — 
that should play proactive roles in making air-quality management decisions: 1) scientists 
and engineers, 2) policy analysts, 3) decision makers, and 4) professional communicators. 

The responsibility of scientists, engineers, and policy analysts is to understand and clearly 
communicate the scientific facts and uncertainties and to describe expected outcomes 
objectively. Deciding what to do involves questions of societal values where scientists, 
engineers, and policy analysts have no special authority. For this reason, the processes of 
risk assessment are very different from the processes of risk management, scientific 
reviewing, legislative policy analysis, and judicial review. 

The proper role of scientists and engineers is to discover and communicate the facts and 
uncertainties associated with the facts. The proper role of policy analysts is to consider the 
facts and associated uncertainties in the light of values held dear by different sectors of 
society. Policy analysts should also provide, for consideration by decision makers, advice 
and counsel about alternative courses of action and the extent to which various sectors of 
society's interests will be affected by each alternative policy option. 

In democratic societies, decision makers are those who are charged by our society to 
make policy decisions - captains of industry, leaders in legislative bodies, executive office 
officials, judges and juries in courts of law, and leaders in public-interest organizations. 

Communication is also crucial in the processes of sound public decision-making — 
communication of research program findings, communication of alternative choices among 
assessment options, communication of certainties and uncertainties. Careful and precise 
communications are needed among all parties in decision making processes — among 
scientists and engineers, policy analysts, and decision makers themselves, between each of 
these groups, and between each of these groups and the public at large. Audiences differ 
widely in their interests, sophistication, and the importance of the subject to them. Scientists 
and policy analysts are selected and educated to discover new knowledge on the one hand 
and to consider alternative courses of action on the other, not necessarily because we are 
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excellent at communications. For this reason, professional communicators constitute the 
fourth important group of actors in the processes of assessment and public decision-making. 

Professional communicators can also serve a vital function in ensuring that members of al 
actor groups do not 'talk past each other'. Perhaps the greatest contribution that 
communicators can make to public policy making is to help members of these other 
professional groups achieve an ideal expressed well in the words of Dr. Daniel Albritton of 
the National Oceanic and Atmospheric Administration — if they will learn to become 
'amphibians' in the mixed-media environment of science and public policy, then critical 
barriers to understanding will be decreased and appropriate use of scientific knowledge in 
public decision-making will be increased. 

Scientific assessment of air-quality management options requires much more than just 
research planning and reporting. A scientific assessment will certainly involve research, but 
assessment is focused on reporting an integrated view of current conditions and future 
projections. Thus assessment includes causes and effects, management options, economic 
and social costs and benefits of different options, and sufficient analysis of future scenarios 
to identify potentially efficient and effective management approaches. A science-based 
assessment should provide information useful for the public, policy officials, and leaders in 
various stakeholder communities whose interests and values will be affected — including 
information on the scientific confidence to be attached to assessment findings. Research and 
assessment should be parallel activities with continuous feedbacks. 

A major science-based assessment requires a written plan that identifies the key questions 
to be addressed, and indicates how measurement data, air-quality models, cost-benefit 
calculations, and other relevant information will be used to address the assessment questions. 
A written plan, developed with extensive comments by inside and outside communities (i.e., 
the assessment team, and the potential users of the assessment information) is an essential 
communication tool. For the assessment team, it establishes the goal and methodologies to be 
adopted, and identifies the context for the work of individual members of the team. For 
assessment users, it provides a clear view of the questions being addressed, the limitations of 
the analyses, and the schedule for the reporting of key findings and recommendations with 
respect to each management option under consideration. 

Specific Comments and Recommendations for Future NAAQS Development Processes 

This past year has provided an unusual opportunity for CAS AC, NCEA, and OAQPS to work 
together in efforts to further optimize the design and organization of Air Quality Criteria 
Documents and Staff Papers. During this one year, in rather rapid succession, CASAC has 
reviewed both planning documents, and reviewed drafts of both criteria documents and staff 
papers for three of the five pollutants presently recognized criteria pollutants. In each case, 
CASAC has been presented with very large documents that require very careful attention from 
the standpoint of many different scientific disciplines. The Members and Panelists in CASAC 
have provided these multidisciplinary perspectives. 

The laws of our country require that this difficult and challenging intellectual work should be 
accomplished periodically (ideally every five years) by scientists, engineers, policy analysts, and 
decision makers who are charged by our society to do their respective parts — leading to 
scientifically sound, policy effective, and socially acceptable decisions in a contentious 
democratic society that often is resistant to change and frequently uses the courts of our country 
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to set demanding deadlines for the development of Criteria Documents, Staff Papers, and the 
promulgation and implementation of Regulations and Rules for air quality management. 

It also has become very obvious that the funds currently being provided for NCEA to 
produce Criteria Documents are not adequate to the task and that substantial increases 
will have to be provided if EPA is ever to get out of the vicious cycle of always having to 
develop Criteria Documents under rushed circumstances that preclude the production of 
optimal-quality and well-focused summaries of the science that is relevant to making wise 
decisions about NAAQS. 

During the past year CASAC Members and Panelists have reviewed and offered carefully 
considered individual and collective advice and counsel about the adequacy of the criteria 
documents, staff papers, and the proposed rules and regulations for ozone and other 
photochemical oxidants, fine and coarse particulate matter, and, most recently, atmospherically 
deposited lead pollutants. 

In all three of these cases, CASAC has done its best to review the documents prepared by 
NCEA and OAQPS and to offer our individual and collective counsel and advice about the 
scientific content, organization, and the scientific objectivity and tone of impartiality of these 
very large criteria documents. 

Beginning in the case of the Criteria Document and Staff Paper for Ozone and Related 
Photochemical Oxidants, a somewhat different organizational structure was used by NCEA. 

The new organizational format called for relatively brief Main Chapters that consist of two 
parts: 

1) A concise summary of "Key findings/conclusions" from earlier assessment documents, 
and 

2) Carefully prepared descriptions of advances in scientific understanding that have been 
developed since the time of the last review and published in more recent scientific 
literature. 

The new structure also calls for development of very detailed Annexes for each Main 
Chapter in which many important advances in scientific understanding are presented in much 
more thorough fashion than in the corresponding Main Chapter. 

The final features of the new structure and organization of Air Quality Criteria Documents 
were development of both an Integrative Summary Chapter and an Executive Summary for the 
whole Criteria Document. The purpose of these two additional parts was to draw together the 
major findings and conclusions of scientific understanding developed within each of the Main 
Chapters and corresponding Annexes and to present in an integrative way the Key Findings and 
Conclusions (from both earlier assessment reports and description of more recent scientific 
advances) and thus provide a maximally useful foundation for the Staff Paper. 

In the words of OAQPS, the purpose of the Staff Paper is to: "provide a critical assessment of 
the latest available scientific information upon which the National Ambient Air Quality 
Standards are to be based. Drawing upon the AQCD, staff in EPA's Office of Air Quality 
planning and Standards (OAQPS) within the Office of Air and Radiation prepares a Staff 
Paper that evaluates policy implications of the key studies and scientific information 
contained in the AQCD and presents the conclusions and recommendations of the staff for 
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standard setting options for the EPA Administrator to consider. The Staff Paper is intended 
to 'bridge the gap' between the scientific assessments contained in the AQCD and the 
judgments required of the Administrator in determining whether it is appropriate to retain or 
to revise the primary and secondary NAAQS." 

Many Members and expert Panelists within CASAC are very pleased with the good sense of 
the revised structure and format of Criteria Documents. We are convinced that these innovations 
in the overarching method of organization of Criteria Documents will better serve the interests of 
the wide variety of audiences that are interested to learn more about scientific understanding of 
each of the criteria pollutants and their effects on both human health and welfare. Thus, many of 
us believe that these innovations in structure should be retained and used in the future in 
preparing other Criteria Documents for other Criteria Pollutants. 

In doing so, it is of course important that the different target audiences for the Executive 
Summary, the Main Chapters of the Criteria Document itself, and the various Annexes be very 
well defined and well understood by all of the staff, consultants, and editors that prepare these 
three different treatments of the same body of scientific knowledge. 

It is even more imperative that the scientific content, objectivity, and tone of impartiality of 
the Executive Summary and the Integrative Summary Chapter of the Criteria Document [and the 
Staff Paper as well!] be consistent not only with the scientific content, objectivity, and tone of 
impartiality of the Main Chapters of the Criteria Document itself, but also with the scientific 
content and objectivity of the more detailed Annexes. Differences in content of these different 
parts of the same Criteria Document [and the related parts of the Staff Paper] should be based 
primarily on their relevancy to their respective purposes and target audiences. Discrepancies in 
scientific content, objectivity, and tone of impartiality in these distinct parts of the Criteria 
Document and Staff Paper will inevitably lead to decreased confidence in the validity and 
reliability of the different parts of both types of documents. Thus such discrepancies must be 
carefully avoided. This will require a larger degree of common understanding among authors, 
consultants, editors, and managers of the Criteria Document and Staff Paper development 
processes than many Members and Panelists within CASAC believe has been achieved to date. 

A useful mechanism for ensuring that there is an effective and concise summary of "Key 
Findings and Conclusions" in each Main Chapter is to require that an Executive Summary be 
prepared for each Main Chapter and that these statement of Key Findings and Conclusions from 
individual Main Chapters be used in constructing both the Executive Summary for the whole 
Criteria Document and in developing the organizational framework for the Integrative Summary 
Chapter. 

In written comments on the Criteria Document for Ozone and Other Photochemical Oxidants 
dated December 2, 2005 I recommended [and affirm here once again] that all authors, 
consultants, editors, and managers engaged in the preparation of Criteria Documents and EPA 
Staff Papers take full advantage of- and use the attached published ''Guidelines for the 
Formulation of Statements of Scientific Findings to be Used for Policy Purposes." 

These guidelines, written in the form of checklist questions, were developed by the members 
of the Oversight Review Board (ORB) of the National Acid Precipitation Assessment Program to 
assist scientists, engineers, and policy analysts dealing with other environmental research and 
assessment programs in formulating statements of scientific findings to be used in policy 
decision processes. As indicated earlier, the distinguished members of the ORB who prepared 
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these guidelines included: Milton Russell, former Assistant Administrator for EPA, Chauncey 
Starr, former Director of Research for the Electric Power Research Institute (EPRI), Tom 
Malone, former Foreign Secretary for the National Academy of Sciences, John Tukey, 
Distinguished Professor of Statistics at Princeton University, and Kenneth Starr, Nobel Prize 
Winner in Economics. 
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GUIDELINES FOR FORMULATION OF STATEMENTS OF SCIENTIFIC FINDINGS 

TO BE USED FOR POLICY PURPOSES 

The following guidelines in the form of checklist questions were developed by the NAPAP Oversight Review 
Board to assist scientists in formulating presentations of research results to be used in policy decision processes. 

1) IS THE STATEMENT SOUND? Have the central issues been clearly identified? Does each statement contain 

the distilled essence of present scientific and technical understanding of the phenomenon or process to which it 
applies ? Is the statement consistent with all relevant evidence that is available in the published literature. Is the 
statement contradicted by any important evidence in the published literature? Have apparent contradictions or 
interpretations of available evidence been considered in formulating the statement of principal findings? 

2) IS THE STATEMENT DIRECTIONAL AND, WHERE APPROPRIATE, QUANTITATIVE? Does the 

statement correctly quantify both the direction and magnitude of trends and relationships in the phenomenon or 
process to which the statement is relevant? When possible, is a range of uncertainty given for each quantitative 
result? Have various sources of uncertainty been identified and quantified, for example, does the statement 
include or acknowledge errors in actual measurements, standard errors of estimate, possible biases in the 
availability of data, extrapolation of results beyond the mathematical, geographical, or temporal relevancy of 
available information, etc. In short, are there numbers in the statement? Are the numbers correct? Are the 
numbers relevant to the general meaning of the statement ? 

3) IS THE DEGREE OF CERTAINTY OR UNCERTAINTY OF THE STATEMENT INDICATED 

CLEARLY? Have appropriate statistical tests been applied to the data used in drawing the conclusion set forth 
in the statement? If the statement is based on a mathematical or novel conceptual model, has the model or 
concept been validated? Does the statement describe the model or concept on which it is based and the degree 
of validity of that model or concept? 

4) IS THE STATEMENT CORRECT WITHOUT QUALIFICATION? Are there limitations of time, space, or 

other special circumstances in which the statement is true ? If the statement is true only in some circumstances, 
are these limitations described adequately and briefly? 

5) IS THE STATEMENT CLEAR AND UNAMBIGUOUS? Are the words and phrases used in the statement 

understandable by the decision makers of our society ? Is the statement free of specialized jargon? Will too 
many people misunderstand its meaning? 

6) IS THE STATEMENT AS CONCISE AS IT CAN BE MADE WITHOUT RISK OF 

MISUNDERSTANDING? Are there any excess words, phrases, or ideas in the statement which are not 
necessary to communicate the meaning of the statement? Are there so many caveats in the statement that the 
statement itself is trivial, confusing, or ambiguous? 

7) IS THE STATEMENT FREE OF SCIENTIFIC OR OTHER BIASES OR IMPLICATIONS OF 

SOCIETAL VALUE JUDGMENTS? Is the statement free of influence by specific schools of scientific 
thought? Is the statement also free of words, phrases, or concepts that have political, economic, ideological, 
religious, moral, or other personal-, agency-, or organization-specific values, overtones, or implications? Does 
the choice of how the statement is expressed rather than its specific words suggest underlying biases or value 
judgments ? Is the tone impartial and free of special pleading? If societal value judgments have been discussed, 
have these judgments been identified as such and described both clearly and objectively? 

8) HAVE SOCIETAL IMPLICATIONS BEEN DESCRIBED OBJECTIVELY? Consideration of alternative 
courses of action and their consequences inherently involves judgments of their feasibility and the importance of 
effects. For this reason, it is important to ask if a reasonable range of alternative policies or courses of action 
have been evaluated? Have societal implications of alternative courses of action been stated in the following 
general form?: 

" If this [particular option] were adopted then that [particular outcome] would be expected." 

9) HAVE THE PROFESSIONAL BIASES OF AUTHORS AND REVIEWERS BEEN DESCRIBED 
OPENLY? Acknowledgment of potential sources of bias is important so that readers can judge for themselves 

the credibility of reports and assessment 
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The Issue of Identical Primary and Secondary Standards 

For many years now, and for many different Criteria Pollutants, EPA has established 
identical primary and secondary NAAQS. In recent years, it has become more and more clear 
from a variety of scientific perspectives, that protection of aquatic and terrestrial ecosystems 
from air-borne pollutants and avoidance of significant deterioration in the quality of scenic vistas 
in both urban and Class I wilderness areas resulting from regional haze will require public 
welfare-based secondary standards that are different in form from public health-based primary 
NAAQS. Thus, many members of CAS AC and the pubhc at-large are looking forward to more 
careful consideration by EPA of secondary standards that will deal more adequately with human 
welfare effects of various Criteria Pollutants. 
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Dr. Bernard D. Goldstein 

Comments of Bernard D. Goldstein, M.D. 
March 16, 2006 



Dr. Vanessa T. Vu 

Director 

EPA Science Advisory Board 

Dear Dr. Vu, 

Tliank you for the opportunity to comment on EPA's review of the NAAQS process. I apologize 
for the delay in responding, in part due to the request coming a little more than a week before 
your deadline, and at a time when I was attending the Society of Toxicology meeting in San 
Diego. 

Let me start with two background statements that frame my approach to these comments. First, 
in my teaching of environmental health policy to both public health students and to law students, 
I routinely present the NAAQS standard- setting process as one that represents an ideal interface 
between science and regulation. 

Second, I have just this past week broken a more than 20-year commitment as a former EPA 
ORD Asst. Administrator of not being publicly critical of EPA. I have done so by being highly 
critical of the way that the EPA Administrator has broken faith with the NAAQS process. 

The strength of the NAAQS process is that it provides an iterative interface between the science 
pertinent to standard setting and the regulatory process. Whatever one believes about the 
scientific appropriateness of the fine particulate standard chosen by Administrator Johnson, there 
is no question that he went beyond the range of the recommended levels reviewed by CASAC, 
and that he did so without the iterative interaction so valuable to the standard- setting process. 
The impression of disregard for this highly successful process undoubtedly will damage the 
credibility of EPA in general, and its NAAQS standards in particular. 

The adverse impact of Administrator Johnson's recent decision goes well beyond the specifics of 
the fine particulate standard. Reviewing the documents from the 1980s that you sent to me, and 
remembering my own tenure as chair of CASAC and then AA of ORD, brings back the many 
discussions at the time of the CASAC process. I am proud of my small role in developing this 
process. I believe that the process as it has existed justifies the enormous number of hours of 
input by the scientific community. This input occurs because we as scientists believe that the 
process appropriately informs the regulator about the extent of the reasonable disagreement 
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among us pertinent to setting the standard. Unfortunately, EPA's recent decision tells the 
scientific community that it is not worth our time to be involved in EPA advisory processes. 

Again, let me emphasize that this unfortunate outcome is irrespective of whether the 
Administrator's decision is appropriate; and let me further emphasize that the Administrator 
could have avoided this criticism simply by asking for additional CASAC review. 

I am not a lawyer, but I have had sufficient experience teaching at law schools and the Federal 
Judicial Center to hazard an interpretation of the existing legal basis for the interaction between 
scientific advisory committees and federal regulatory agencies. The recent quotes from EPA that 
they do not have to listen to the advice of CASAC are of course correct - CASAC is purely an 
advisory committee. But I suggest that EPA carefully review the API v. Costle decision (42 
U.S.C. 7607 (d)(8)). In that decision about the 1979 ozone standard the court indeed affirmed 
that EPA need only hear its scientific advisers, not follow their advice. However, the court 
found that EPA erred in never having submitted the ozone standard for consideration by its 
advisory body. In this case the court did not find that EPA's error raised a substantial likelihood 
that the rule would have been significantly changed, so they found in EPA's favor. Given the 
current CASAC response to EPA's fine particulate decision, it is not certain that a court will be 
so forbearing in this case. In essence, EPA may have illegally made an important regulatory 
decision without obtaining advice as to its scientific soundness from its congressionally- 
mandated scientific advisors. 

Effective protection of public health and the environment is heavily dependent upon the best 
quality science and the effective translation of this science to those responsible for regulatory 
decisions. The process developed for NAAQS standards has been highly successful and has been 
a model for how science and scientists can and should be used to provide credible advice that can 
be translated into effective and defensible regulatory standards. Tampering with a process that 
has been so effective should not be done lightly. 

I hope the above is helpful, and I would welcome further involvement in discussions about the 
NAAQS process. 



Bernard D. Goldstein, MD 

Professor of Environmental and Occupational Health 

Graduate School of Public Health 

University of Pittsburgh 
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Dr. Rogene F. Henderson 

Comments by Rogene F. Henderson 
March 6, 2006 

Key Questions for the Review of the Process for Setting NAAQS 

Timeliness of the NAAQS review process 

• What are your views on the timeliness and efficiency of the current process for both 
EPA's and CASAC's reviews of the air quality criteria and the NAAQS, in terms of the 
time that is spent between the start of the review and the publication of the Agency's 
proposed decisions on the standards? 

I am in my second year as Chair of CASAC and as such, I do not have the long-term 
experience that many others have had in this process. This puts me at a disadvantage in 
knowing some historical data, but I have the distinct advantage of looking at the CASAC 
review as part of the NAAQS process from a fresh viewpoint. I think that people on the EPA 
staff and on the CASAC both work very hard on the review process, and it is well worth 
considering how to make the process more efficient. 

One thing that seems to slow things down is what I call a "ping-pong" review process. I 
chaired the National Research Council's Committee on Toxicology (COT) for eight years 
and we once had the same problem. A subcommittee of COT was reviewing documents on 
recommended levels of exposure for an agency. The agency was under great pressure to meet 
deadlines for getting us the documents to review. Sometimes the documents were not really 
in good shape, but they had met their deadline. The COT would then have many comments, 
both editorial and scientific. The document would then go back and forth between the 
agency and COT until a satisfactory draft was obtained. Some of the problems I observed in 
chairing COT, I have observed in a somewhat magnified fashion in the CASAC process. The 
ping-pong process begins when the Agency is rushed to meet a deadline and submits a less 
than optimal document to the advisory body to review. The CASAC goes over the document 
carefully, commenting not only on scientific matters but on editorial points and asks to see 
the revised draft again. This process may go through several iterations until the CASAC is 
satisfied with the draft. If such an approach continues, the initial drafts submitted from the 
Agency may become more and more premature, because they have to meet a deadline and 
they know they will get the benefit of a good outside review before the draft is finalized. 

To prevent that type of cycle from occurring I suggest the following: 

1. All documents sent to CASAC for review should be the Agency's "best and final" 
version of the document. It would be more time saving to miss a deadline than to submit less 
than adequate documents to CASAC for review. The submitted documents should have been 
thoroughly reviewed in-house and should be in a form that makes it easy for CASAC to say 
they do not need to see it again. 
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2. Adequate staffing should be assigned to the task to allow a reasonable chance that 
credible documents can be produced in the time allowed. 

3. The CASAC should provide clear scientific advice, but not editorial advice. It is a 
waste of valuable expert scientists' time to have to make the EPA documents readable. The 
NRC has an excellent editor that provides this type of review for NRC documents and I think 
the EPA should be responsible for the same type of editorial review of their documents. 

4. 1 found a sense among several CASAC members that the CASAC is responsible for 
approving the proposed standards rather than giving advice and recommendations . The 
Agency should make clear to CASAC what they require in terms of scientific advice and 
what they consider to be policy issues, on which they do not need advice. The line between 
science and policy is not always apparent, and this difference should be made clear in the 
charge questions given to CASAC. Both the Agency and the CASAC have the same goal — 
to protect public health and the environment. The relationship between the Agency and 
CASAC should be a collaborative one, in which both groups work for the greatest good. The 
scientists can provide excellent expert advice and are obliged by law to recommend the range 
of standards that would be appropriate. In the end, however, the Administrator of the EPA 
has the responsibility to decide what the standards will be. If policy plays a major role in that 
decision, the Administrator should make the policy choices clear to the public and to the 
CASAC. There should be no surprises. 

• Can you identify structural changes to the process and/or key documents (e.g., the 
Criteria Document, Stajf Paper, Risk Assessment) or changes in the Agency 's 
management of the process that could shorten this timeframe while preserving an 
appropriately comprehensive, transparent and policy-relevant review and allowing 
adequate opportunities for CASAC review and advice and for public comment on these 
documents? 

I think the process can be broken down into three major parts: assembly of the pertinent 
literature, development of an integrative chapter describing this literature, and development 
of the staff paper. The first two parts of the process are given in the CD. I think there is a 
more efficient way to accomplish the first part. 

1 . The literature review part of the CD could be completed without a face to face 
meeting. At present the CD is an unwieldy document, a compendium of all research done on 
the criteria pollutant of interest. The CD is a valuable resource and has been used by many 
students and agencies as a reference work. Progress has been made in making the document 
more readable by putting the most critical, new material in the main text and the rest in 
appendices. The role of CASAC is to look over this literature review and advise whether all 
the important studies have been included and if the Agency has interpreted them correctly. 
Recommended change: Once the literature review has been completed, a draft of each 
chapter could be submitted to a subgroup (2 or 3 people) of the CASAC panel for their 
review. Needed alterations in each chapter could be addressed via a teleconference so that by 
the time the full CASAC panel meets, all of the literature review chapters are acceptable to a 
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subgroup of CASAC. Then at the face to face meeting of the full panel, very little time would 
be required to describe the main points of each chapter so other members of the CASAC 
panel and the public will be well informed. This would allow more time to discuss the critical 
integrative chapter of the CD. 

2. The integrative chapter of the CD should be the major point of discussion at the first 
face to face meeting of the CASAC panel. This is the point of departure for the subsequent 
development of the Staff Paper. 

3. The Staff Paper is the critical document and, in my brief experience, has been well 
written. This is the document for which the CASAC expertise is most needed. I would 
suggest that meetings to discuss the Staff Paper might be extended to 2 and a half days to 
allow more discussion of this important piece of work. At the request of the CASAC, more 
time should be allowed for presentations by scientific experts who may not be on the panel. 

4. The public comment period and the transparency of the process should be maintained. 

Consideration of the most recent available science 

• To enhance the Agency 's ability to take the best and most recent available science into 
account in making decisions on the standards, can you suggest changes in the process 
and/or key documents that could shorten the time between the presumptive cutojf date for 
scientific studies evaluated in the review and reaching proposed decisions on the 
standards, or that could otherwise facilitate appropriate consideration of more recent 
studies ? 

I would suggest that critical new studies should be presented to CASAC for review and 
included in the Staff Paper up until the Staff Paper is finalized. In the time between the 
completion of the Staff Paper and the proposal of revised standards, only a study that might 
make a large difference in the standard settings should be considered and should be reviewed by 
CASAC. This would have to be a judgment call. It would not be appropriate to base decisions 
on papers that have not been reviewed by CASAC. 

Distinctions between science and policy judgments 



• 



• 



Recognizing that decisions on the standards, while based on the available science, also 
require policy judgments by the Administrator, what are your views on how clearly 
scientific information, conclusions, and advice are distinguished from policy judgments 
and policy recommendations on the standards throughout the review process? 

Can you suggest changes in the process and/or changes to the format and contents of key 
documents that would help to make these distinctions clearer? 

I think this is a difficult distinction to make and it is not clear to me where to draw the 
line. It would be helpful if this distinction were clearly drawn in the initial charge questions. In 
other words, spell out where you need science advice and what territory is policy driven. 
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Identifying, characterizing, quantifying, and communicating uncertainties in scientific 
information 

• Recognizing the importance of characterizing and clearly communicating the 

uncertainties in the science and quantifying uncertainties in exposure and risk estimates 
as explicitly as possible, what are your views on any changes in the process and/or 
changes to the format and content of key documents that might facilitate a more 
complete, quantitative, and policy -relevant characterization of uncertainties? 

How one deals with the uncertainties is a policy issue. One can say that a lot of 
uncertainty suggests being more conservative to be sure we are "safe." Another policy might be 
that a large amount of uncertainties means that we cannot select appropriate levels until we have 
more information. In any case, the amount of uncertainty should be fully addressed and central 
estimates should be given as well as the upper and lower confidence limits. Again, the policy 
decisions made should be explicit and clearly stated in public. 
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Dr. Philip K. Hopke 



Comments on the NAAQS Review Process 

Philip K. Hopke 
February 24, 2006 

The Clean Air Act (Amended) calls for several things with respect to CASAC's role in the 
process of setting ambient air quality standards. It calls for the Administrator to: 

(2)(A) The Administrator shall appoint an independent scientific review committee composed of 
seven members including at least one member of the National Academy of Sciences, one 
physician, and one person representing State air pollution control agencies. 

(B) Not later than January 1, 1980, and at five-year intervals thereafter, the committee referred to 
in subparagraph (A) shall complete a review of the criteria published under section 108 and the 
national primary and secondary ambient air quality standards promulgated under this section and 
shall recommend to the Administrator any new national ambient air quality standards and 
revisions of existing criteria and standards as may be appropriate under section 108 and 
subsection (b) of this section. 

(C) Such committee shall also (i) advise the Administrator of areas in which additional 
knowledge is required to appraise the adequacy and basis of existing, new, or revised national 
ambient air quality standards, (ii) describe the research efforts necessary to provide the required 
information, (iii) advise the Administrator on the relative contribution to air pollution 
concentrations of natural as well as anthropogenic activity, and (iv) advise the Administrator of 
any adverse public health, welfare, social, economic, or energy effects which may result from 
various strategies for attainment and maintenance of such national ambient air quality standards. 

It should be noted that there is an explicit requirement for CASAC to "recommend to the 
Administrator" the new standard. 

It is obviously better to have the Committee and its larger specific pollutant panels come to a 
well defined consensus with respect to its recommendations. That is the value of the formal 
"closure" procedure and this approach should be reinstated. 

Timehness of the Review Process 

Comments 

Let us look at what went wrong with the review of PM standard that was completed last year. 
The first problem was the presentation of documents that were clearly not ready for review. The 
first draft CD and SP should have never been presented to the Panel. If the agency wants 
extended feedback to use in writing a real first complete draft, then they should consider 
providing a white paper that outlines the major issues they think exist and their plans for dealing 
with them. In both cases, these incomplete and weak documents started the process off on the 
wrong foot. 
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The format of the CD was also a problem. The placement of the detailed description of 
individual studies and similar levels of descriptive material causes the panel to focus on the 
minutiae instead of the main take home messages. What the NCEA staff really needs to do is to 
write the summary chapter first carefully recognizing the major take home messages and then 
use annexes to buttress the case. This was largely the approach in the ozone document and it can 
be seen how much easier it is to review this document. This is not simply a matter of a less 
complex subject, but being able to focus the review of the content that matters most. 

We lost a year because of the GAM problem. The staff built much of its case on the 
epidemiology studies that used the GAM protocol and thus, it was not possible to proceed with 
the review of the epidemiological evidence without those studies being redone, described and 
reviewed. We arranged with the Health Effects Institute for an expedited review and given the 
magnitude of the effort needed by the original investigators, the reviewers and the NCEA staff, 
completion of this effort within a year was really about as timely as it could be done. It is not 
clear to me that this problem was adequately presented to the court in order to obtain a sufficient 
time period following the completion of the HEI review to complete the revisions of CD and the 
remainder of the process. The attempt to short circuit the process with the first draft SP was not 
helpful as it left too much out and thus, left too many openings for criticism. It therefore was 
hard to forge the consensus that provides a critical symbol of authority for the Agency to act. I 
believe that if the Agency had waited a few months longer to provide a more fully reasoned and 
complete SP, we would have come to closure in a more rapid manner. 

I believe that the lack of closure on the staff paper will provide an additional point of leverage 
for the potential litigants to argue that there was not agreement on the scientific basis for the 
standard as had been the past practice. 

Recommendations on Timeliness: 

I strongly suggest the format of the CD be like that of the ozone document. Focus on the 
synthesis first and foremost and do not leave it to be an afterthought. Do not release documents 
because they want to have something out. Wait until it is ready. The time can be made up later 
because there will be fewer criticisms and more willingness to compromise on the criticisms if 
the Panel feels that a real effort has gone into crafting a complete, comprehensive and well 
reasoned document. This applies to both the CD and SP. Reinstate the closure process so that 
there is a clear and final approval of the document. I recognize that only the CASAC members 
have standing, but again having the record indicate that all of the panel can agree that the 
document adequately presents the scientific basis for the policy decisions will be valuable to the 
Agency as it proceeds through the full process of promulgating and implementing standards. 

Consideration of the Most Recent Available Science 

There has never been a prohibition of inclusion of seminal new work that would significantly 
alter our view of the pollutant in question in any manner that significantly affects the setting of 
the standard (indicator, concentration, time interval, statistical form). In general, the literature 
tends to be quite incremental and although additional papers generally will add strength to the 
conclusions obtained in the CD and following on into the SP, it is going to be a very rare 
occurrence when such a paper appears that it really changes directions. If such a paper appears, 
then it can certainly be included in the body of the document. There are a variety of ways that a 
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quick review of such a last minute addition could be made through a teleconference or even an e- 
mail polling of the relevant Panel. Otherwise, it is important to set a fixed cut-off date or the 
document will constantly be subject to revision. However, even here there could be additions to 
the annexes. If the CD stays away from referencing individual papers and only looks to 
summarize the substance of the subject derived from the body of literature in the annexes, then it 
is possible to add incremental material in the annex with fewer problems and such additions are 
easy to track and to have the panel approve. 

Distinctions between Science and Policy Judgments 

In this case I hearken back to the law which asks the Committee to recommend a standard. 
CASAC has typically left the recommendation to the staff through closure on the SP. Now since 
closure has been eliminated, it becomes incumbent on the Committee to make a formal 
recommendation and this will clearly include more than the science. The loss of closure has 
helped to blur the line between scientific advice and clearly leads to the Committee taking a 
more active policy role. I would suggest that this direction may take the Panels in directions 
that, in fact, take more time to come to consensus and thus, again I argue for a return to the 
closure process where there was an implicit recommendation of the standard through a consensus 
acceptance of the SP recommendations. 

Identifying, Characterizing, Quantifying, and Communicating Uncertainties in Scientific 
Information 

This has been a major role of the CASAC panels since there is enormous pressure put on the 
NCEA staff to come up with unequivocal statements that the regulatory staff can use to support 
their decisions with respect to regulations. It has typically been one of our major criticisms of 
the documents that they do not adequate reflect the degree of uncertainty in the science often 
because of selective citation of papers that support one direction versus another. We do not want 
the documents to reflect more uncertainty than is present in the literature, but we also do not 
want less. Obviously conveying uncertainty can best be done quantitatively if numerical values 
can be provided. More often than not, it is necessary to describe the state of the science 
qualitatively. The key again is the integrative summary. If we make the integrative summary 
the body of the CD with the supporting evidence in annexes and write that first (or at least 
outhne the major points to be made), then everyone can focus on the key issues of what we 
know, what we do not know and how well we know what we do know. Such a clear statement of 
the related science would provide a better basis from which to build the policy review and 
recommendations. 

Summary 

The best advice I can provide is to do more work up front in a more effective manner. The 
ozone CD provides the start for a template for how to do things. It would have been even better 
if the welfare portion of the document had been as effectively written as the rest of the initial 
draft. Getting off on the right foot and focusing on the key, bottom line issues instead of the 
minutia will provide a more effective and efficient approach to writing, reviewing, revising, and 
closing on these documents. It also provides an opportunity to put additional literature in the 
annexes without as much hassle as when they become part of the main document. 
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Dr. Morton Lippmann 

NAAQS Process Comments 

Morton Lippmann 
February 24, 2006 

Background and Credentials 

I began my service to CASAC as a Core Consultant in 1980, became a statutory member in 
1982, served as Chair from 1983 through 1987, attended CASAC meetings as a member of the 
SAB Executive Committee from 1987 through 2001, and have served as a member of CASAC's 
PM and Ozone Panels until the current year. 

I wrote a review and commentary entitled "Role of science advisory groups in establishing 
standards for ambient air pollutants" that was published in Aerosol Science and Technology 
6:93-1 14 (1987). Many of the comments and recommendations therein are still relevant today. A 
copy was provided by Harvey Richmond to Fred Butterfield, and it was attached to Fred's memo 
to current and former members of CASAC of Feb. 24. 



Endorsement of Dr. Mauderly's Comments 

I have read and fully endorse the Feb. 21, 2006 comments made by Dr. Joe Mauderly on the 
NAAQS Process, and will not elaborate on the issues that he addressed. The comments that I 
offer below supplement and extend the issues that need to be addressed by the current members 
of CASAC in the recommendations that they will be submitting to AA's Wehrum and Gray by 
April 3. 



Can the Process for Setting NAAQS be Strengthened? 

The easy answer is of course it can, and I will address how it can in text that follows. However, it 
is important that any changes made in the process do not weaken the long-established integrity, 
objectivity, and credibility of the process to the scientific community and interested stakeholders. 
This needs to be explicitly considered in light of the recent changes in SAB Staff management of 
CASAC's modus operandi in relation to its demands for discontinuing the issuance of a formal 
'CASAC closure letter' on Air Quality Criteria Documents (CDs) and Staff Papers (SPs) from 
the CASAC review process. This management decision was unwise, and has already resulted in 
CASAC initiatives to offer public comments after EPA's completion of final versions of the 
latest PM CD and the Administrator's Proposal for PM NAAQS. I will therefore first address 
the need for CASAC to regain its ability to fulfill the role mandated by the Clean Air Act 
Amendments of 1977 to review NAAQS criteria, and the mandate of the Environmental 
Research and Development Demonstration Authorization Act of 1977 for SAB to review 
Standards. CASAC has always issued its closure letters directly to the Administrator without 
oversight by the SAB Executive Committee. Its independence is therefore compromised by the 
imposition of SAB Staff management decisions on its process. 
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The parts of the NAAQS setting process that can and should be strengthened are the parts played 
by NCEA and OAQPS, and CAS AC can and should assist these EPA offices in doing so. The 
long gestation and document preparation times of CDs and SPs for CASAC review account for 
the long, drawn-out time scales of NAAQS reviews, not the times attributable to CASAC review 
and preparation of its reports and letters. 

The most urgent need is for NCEA to prepare a first draft of each CD that is really ready for 
'prime time'. Before preparing a first public review draft, NCEA needs to decide which issues 
are most critical to standard setting, and who among its staff and outside consultants can 
effectively address them. It then needs to utilize expert workshops and/or CASAC consultations 
to identify the literature and other information sources that are germane to these issues. Only 
then should it prepare or commission draft chapters or sections thereof. This first draft should 
include interpretive summaries of the health and welfare issues even if they remain less than 
complete. Informed CASAC commentary on these integrative chapters can help to ensure that 
any necessary feedback to the authors can lead to the incorporation of appropriate revisions, or 
the fiUing of critical knowledge gaps, in the next, and presumably final draft. If this approach is 
rigorously followed, there should only be no need for a third draft for CASAC review. 

There is an urgent need for the development of a better and more consistent vocabulary for new 
CASAC Panel members and document authors before draft chapters are prepared and reviewed. 
Terminology that needs to be standardized and used consistently includes: 

* sensitive subgroups: How large and/or how extra-sensitive does a definable group have to be 
to warrant the setting of a NAAQS specifically designed to protect them against adverse health 
effects arising from their exposures to ambient air pollutants. 

* adverse health effects: What is an 'adverse' health effect? For the hmited number of Criteria 
Pollutants, there should be pollutant- specific effects that are defined in advance of the CD 
preparation. Is there a degree of adversity that triggers the need for protection by the enforcement 
of a NAAQS? 

* susceptible individuals: For those relatively few people whose special susceptibility leaves 
them unprotected by NAAQS designed to protect sensitive subgroups, how can EPA and state 
and local agencies provide adequate guidance on measures to avoid harmful exposures. 

* adequate margin of safety: There is a widespread recognition that, for at least some criteria 
pollutants, i.e., PM, 03, and Pb, the available literature provides no evidence for the existence 
population-based threshold concentrations. Thus, there is a need for a new operational definition 
of a NAAQS that provides an adequate margin of safety. A 'policy' decision is needed on a level 
of public health risk that is acceptable when a NAAQS is enforced. 

* population based thresholds: In the absence of evidence for population based thresholds, 
there is a need for a 'policy' decision on the most prudent course to follow for risk assessment. Is 
there an alternative to the assumption that a linear or other smoothed curve that fits the best 
available epidemiologic data should be used? If so, it needs to be made explicitly. 
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* acceptable level of population risk: A 'policy' decision is needed for the ground rules on 
what constitutes an acceptable level of population risk when the health effects data are consistent 
with non-threshold population-based linear or curvilinear relationships. For example, is 3 days of 
life- shortening of a chronically-ill senior citizen due to a peak in 24-hr PM2.5, or the loss of 1 or 
2 I.Q. points in a Pb-exposed child, acceptable? 

The Interface between Science and Policy 

CASAC has recognized, and must continue to recognize that there is a clear need for it to 
provide advice and guidance to the Administrator and the Congress on the science relevant to the 
setting of NAAQS, and must avoid, to the extent possible, on policy decisions. The difficulty in 
drawing such distinctions is evident if one considers my above stated needs for standardization 
of key elements of the terminology that CASAC confronts when dealing with NAAQS issues. 
Each of them approaches or crosses the line between science advice and public policy issues. 
The choices that must be made on defining or clarifying policy relevant to meeting the legislative 
mandates must be made by the Administrator and/or by Congress through revisions to 
established Acts, and CASAC 's role must be limited to highlighting the issues at the science- 
policy interface and the scientific knowledge that informs these issues. 
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Dr. Joe Mauderly 



Comments on the NAAQS process 

Joe Mauderly 
February 21, 2006 

Timeliness of review process 

It is extremely important to both refine and speed up the review process. Not only has it become 
embarrassingly common for the process to lag such that deadlines are now routinely set by legal 
actions, but that mode of operation easily becomes an excuse for failing to make the effort to 
produce the best product, or to limit CASAC review in the late stages of the process. The law 
says that NAAQS pollutants are to be reviewed at five-year intervals. The law does not say what 
the review must consist of, or how it is to be done. Either the current approach or the law needs 
to be changed. In fact, it is entirely possible to review the pollutants every five years (assuming a 
will to do so). What is not possible is to do so using the current approach. 

A key improvement would be the development of better documents before they are given to 
CASAC and the public to review. My experience suggests that much of the time for review is 
incurred by the failure of authors to do a good job the first time. There is too much reliance on 
CASAC to edit documents, because of either the reluctance or inability of EPA managers and the 
original authors to review and optimize them before they are distributed. For example, it is not 
rocket science to determine whether or not a "synthesis" of important information at the end of a 
chapter is indeed a good synthesis of the foregoing material, yet it is too often left to CASAC to 
state the obvious before a decent synthesis is written. The same holds for chapters that are 
intended to integrate information from foregoing chapters. Because CASAC appropriately 
attempts to hold documents to a high standard, it will serve an editorial function by default, but it 
should not be so necessary. 

One of the reasons given for the recent (apparently successful) move by EPA to relegate CASAC 
to a reviewer, rather than an approver, of documents is that it slows the process. That is pure 
balderdash. I cannot recall a single instance over my 15 years of experience with the Committee 
that CASAC was truly the root cause of significant delay. On the other hand, I can recall 
multiple instances in which, if CASAC had not the prerogative to "close" on documents, EPA 
was clearly on track to ignore scientific advice and move forward with inadequate documents or 
incorrect conclusions. If CASAC points out deficiencies that need to be remedied, it is not 
CASAC that is delaying the process. 

There is no way to substantially shorten pollutant reviews unless a different, and more parallel 
process is adopted. It could be speculated that a CD development process more akin to the NRC 
committee process might offer possibilities. That process involves engaging scientific experts in 
drafting, refining, and developing consensus about documents that review equally difficult 
scientific issues. The process could include members of CASAC, as well as other subject matter 
experts (the present "Panels" set precedent). It may be that such a process could result in 
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development of a more concise review and interpretive document than the present CD. 
Voluminous material could be cataloged and summarized in tabular form at the committee's 
direction, by lesser credentialed EPA staffers working with the committees. This is done at NRC 
- staff often does the bulk of the "busywork" under the guidance of committee members. Just as 
NRC has many committees working simultaneously, EPA could have committees working on 
multiple pollutants in a parallel manner, rather than the largely linear current process. Or, of 
course, you could just turn the CD process over to NRC - I'd guess that the Academy would not 
turn down the contract. 

Consideration of the most recent science 

This is fundamentally impossible in the strictest sense. "Science" emerges daily. In order to 
avoid paralysis, it is critical to develop, state, and adhere to a policy for cut-off of published 
information feeding into the CD and SP. However, special circumstances will inevitably arise in 
which post-CD information is of such novelty and importance that it is illogical (if not 
unconscionable) to disregard it in the final promulgation. That circumstance is not as frequent as 
most of us researchers like to presume; new studies reinforcing already- stated findings or 
conclusions do not qualify. Only information that clearly confirms exposure-response 
relationships for new effects of pollutants or proves markedly different estimates of known 
effects would qualify. 

There may be an opportunity for improvement here, if it could be managed well. Assuming a 
sequence similar to the present (CD followed by SP, followed the proposed standards), either 
EPA or CASAC could assume responsibility for monitoring new published findings, and 
screening them for publications that truly alter our understanding of exposure-effects 
relationships (for either primary or secondary standards). CASAC could give a quick opinion 
(i.e., within weeks, not months) as to whether or not the information met the impact criteria. This 
process could be done by distribution of papers and conference calls. 

Distinctions Between Science and Policy Judgments 

This takes discipline, and perhaps more than we've been wiUing to exert. As long as we have our 
present approach to regulation, there is, in fact, a distinction between science and policy. Neither 
scientists nor policy makers want to draw the line, or to define it or admit to it. CASAC meetings 
are rife with discussions about how its pronouncements will affect policy, and scientist advocates 
(on CASAC and its panels, as well as others) game the system to achieve their ideological policy 
goals. When EPA proposes or promulgates standards, it is reluctant to state clearly how science 
and policy enter into the decision - it wants to portray that all is based on science. These 
behaviors are absolutely understandable - most scientists are convinced that they know what's 
best for the country, and EPA Administrators don't want to admit to any motive other than the 
"best science". 

The problem is that the "policy" factors might logically be raised, along with the science, in the 
SP, but then CASAC would be placed in the position of reviewing policy. As appealing as that 
might be to some members and panelists, that does not seem to be their statutory role (and is 
seldom their expertise). To adequately review "policy" issues would require an expanded 
spectrum of expertise on CASAC. 
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One possibility is to constitute either a CASAC-linked group or some independent, but 
conceptually similar peer review group to deal with policy. That is a remote possibility indeed! 
No administration on either side of the isle would welcome policy by independent expert 
consensus. 

At present, my only suggestion is that the Administrator make explicit (much more so than at 
present) just how science and policy separately bore on the proposed standard, and how the two 
were integrated. That is asking for more transparency than agencies and administrations (of any 
political stripe) are likely to be willing to yield. To the extent that non-science (?) policy impacts 
could be made clear, it might reduce the tendency on the part of scientists to conclude that they 
just haven't yelled loudly enough. 

Identifying, Characterizing, Quantifying, and Communicating Uncertainties in Scientific 
Information 

There needs to be a more explicit characterization of uncertainty in estimates of causality and 
exposure-response relationships (again, for both primary and secondary standards). At present, 
assessments of "uncertainty" are almost completely focused on the mathematical uncertainty of 
effects estimates (i.e., confidence intervals on measurements of exposures and effects). This is 
important of course, but I would like to see a more rigorous discussion of "certainty" in a broader 
sense. For example, how do the magnitudes of health effects of air pollution rank in comparison 
to other voluntary and involuntary health risks? Because air pollutants seldom, if ever, exert 
novel effects, what portion of the total public health effect is plausibly attributable to a pollutant 
(or to pollution)? What do we know about the relative benefits, and cost-benefit relationships, of 
different approaches to reducing health burdens that are exerted in part by air pollution? I care 
not that these issues might not fall within many folks' definition of "scientific information", or 
that EPA is not supposed to take cost into account in promulgating standards (does any thinking 
person actually believe that they shouldn't, or don't?). We delude ourselves and miss 
opportunities to inform policy makers and promote a rational public understanding of risk if we 
continue to view the "uncertainty" issue as solely one of statistical methodology and data quality, 
while advocating for the special importance of the particular effects (no pun intended, but if the 
shoe fits -) by which we make our living. 
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Dr. George T. Wolff 

Comments on the NAAQS Review Process 

George T. Wolff 
March 3, 2006 

I welcome the opportunity to provide comments to Mr. Wehrum and Dr. Gray on the NAAQS 
review process. I have been an active member of the SAB for the past twenty-one years, and 
participated on numerous SAB committees. During that time, I also participated in seven 
NAAQS reviews and was chair of CASAC for four of them. The lengths of the reviews ranged 
from three years for CO, NO2, SO2, and PM (1994-1996 review) to six years for the recent PM 
review. 

While I will address Mr. Wehrum and Dr. Gray's specific questions, I would first like to discuss 
some historical aspects of the reviews that I believe have relevance to the review process. The 
previous PM review was completed within three years (1994-1996) under a court-ordered 
deadline. So it is possible to complete a review and come to closure within a three year period. 
However, a consensus was not reached in that review on the concentration level of the standards. 
I refer you to the table (which I have appended) in the June 13, 1996 closure letter (EPA-SAB- 
CASAC-LTR-96-008). Individual Panelists' recommendations for the annual PM^.s standard 
ranged from 15 to 30 |Jg/m^ and for the 24-hour standard from 20 to 75 |Jg/m^ . The closure letter 
explains this "diversity of opinion": 

"The diversity of opinion also reflects the many unanswered questions and uncertainties 
associated with establishing causality of the association between PMi.s and mortality. 
The Panel members who recommended the most stringent PM1.5 NAAQS, similar to the 
lower part of the ranges recommended by the Staff, did so because they concluded that 
the consistency and coherence of the epidemiology studies made a compelling case for 
causality of this association. However, the remaining Panel members were influenced, to 
varying degrees by the many unanswered questions and uncertainties regarding the issue 
of causality. The concerns include: exposure misclassification, measurement error, the 
influence of confounders, the shape of the dose-response function, the use of a national 
PM2,5/PMio ratio to estimate local PMi.s concentrations, the fraction of the daily mortality 
that is advanced by a few days because of pollution, the lack of an understanding of 
toxicological mechanisms, and the existence of possible alternative explanations." 

In contrast to the 1994-1996 review, the 1999-2005 review took 6 years, was not allowed to seek 
closure on the documents, but achieved a majority opinion in support of lowering both the annual 
and 24-hour standards. There were some important differences in the process that lead to the 
different outcomes. 

There were two important reasons why the review took so much longer. The first was the GAM 
software issue which was beyond the control of the Agency or the Panel, and this added at least a 
year onto the process. A second more important reason is that the documents (the Criteria 
Document (CD) and Staff Paper (SP)) given to the Panel to review were far inferior to the ones 
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given to the previous panel. In the 1994-96 review, the Agency acknowledged in the documents 
the numerous and large uncertainties that caused CASAC's "diversity of opinion," and as a result 
produced more objective documents. Even though some members disagreed with the Agency's 
interpretation of the data and EPA's ultimate recommendations, they approved of the documents, 
because they contained relatively balanced discussions of the uncertainties. 

The recent review began with the Agency attempting to minimize the uncertainties by selectively 
citing new studies (in whole or in part) that supported their 1997 decision and ignoring other 
studies (or other results in the cited studies) or rationalizing results they did not like away. This 
is the main reason why the review took so long. Drafts were sent back for revisions not for 
significant technical errors but to remove biases and achieve more balance. Each subsequent 
draft was more balanced, but numerous biases still remained in the final documents. A closure 
requirement could have further reduced the biases. I say more on the closure issue later. 

A second significant difference between the reviews is the composition of the Panel members. 
In the 1994-96 review, there were a number of Panel members who were skeptical that the 
epidemiology studies demonstrated cause and effect including one biostatistician and one 
epidemiologist who were not authors of the studies that found statistical links between PM and 
health endpoints. As a result, the Panel expressed "a diversity of opinion." 

When the new Panel was formed, most of the Panel members who supported a causal role in 
1996 were invited back to be on the new panel. Most of the skeptics were not. Instead they were 
replaced by individuals that, on the balance, were more supportive of the Agency's position. In 
fact, by the time the Panel concluded the review, seven out of 22 members had been authors of 
papers that purport causality. No epidemiologist or statistician who questioned causality was a 
member of the Panel. This lack of balance on the Panel predetermined the outcome of the 
review. 

Timeliness of the Review Process 

As indicated above, many of the previous reviews were completed in a three year time-frame, 
which I consider to be timely. However, the process can still be improved. The limiting factor 
here is the quality of the documents. Efforts must be made to produce objective, unbiased 
documents. Brevity needs to be a goal. There has been much discussion over the years over 
how the CD, in particular, needs to be shorter. A template needs to be developed and followed 
that stresses brevity and objectivity and maximizes the use of tabular summaries of the studies. 

The recent decision by the Agency to eliminate the need for CASAC closure will shorten the 
process, but, in my opinion, was a bad decision, and I fear that quality will suffer. The iterative 
review process leading to closure gave the Agency incentive to produce a document that CASAC 
would approve. Removing that incentive could lead to inferior products. 

A word about public comments - Over the years there have been numerous excellent scientific 
comments produced by various organizations. Unfortunately, they typically arrive a day or two 
before the CASAC meeting, which gives the members insufficient time to digest them. I suggest 
that there be a cutoff date of ten days to two weeks before the meeting. As of now, relevant 
public comments on the CD and SP go into a black hole and are only addressed if EPA wants to 
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or a CASAC member or two push for it. Some Agency response to the public comment 
documents should be prepared and provided to CASAC. 

Consideration of the Most Recent Available Science 

The present PM review represents the extreme because of the length of the review. The cutoff 
date was adhered to with the understanding that exceptions would be made if we all agreed that a 
new study was exceptionally important, and, of course, we had to wait for the GAM re-analysis 
studies. Aside from the GAM re-analyses studies, there were several additional papers 
considered, but EPA only included those supportive of their position and excluded others that 
members of the Panel suggested. Thus, there is a need for explicit criteria as to which studies 
qualify as "exceptionally important." 

Distinctions between Science and Policy Judgments 

The selection of a particular level for a standard is a policy judgment. CASAC s job is to insure 
that the range, form and averaging time recommended in the Staff Paper have a scientific basis. 
In questioning the recommendations in the January 17, 2006 NPRM, CASAC has clearly 
overstepped their boundaries and ventured into the policy arena. 

Identifying, Characterizing, Quantifying, and Communicating Uncertainties in Scientific 
Information 

The Agency has not done an adequate job here. In the PM review, only the statistical 
uncertainties were considered. The Agency completely ignored the larger uncertainties 
associated with various assumptions made by individual investigators including, but not limited 
to, the selection of the appropriate model, choice of temporal smoothing functions, control of 
confounders including meteorological parameters, adequacy of exposure metrics, selection of lag 
structures etc. It is not that the Agency is unaware of these uncertainties; they just choose to 
ignore them in the risk assessment. When the GAM re-analyses were being conducted, some of 
the investigators conducted sensitivity analysis by varying some of these assumptions within 
plausible limits. They found that they got a spectrum of results, both positive and negative. This 
led the HEI Special Panel of their Review Committee to write in their commentary: 

"Neither the appropriate degree of control for time in these time-series analyses, nor the 
appropriate specification of the effects of weather, has been determined. This awareness 
introduces an element of uncertainty into the time- series studies that has not been widely 
appreciated previously." 

To insure that such uncertainties are incorporated into the Agency's SOP will require high level 
intervention from senior EPA management and the selection of individuals to CASAC who have 
an appreciation of the importance and significance of these uncertainties. 
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From June 13, 1996 Closure Letter (EPA-SAB-CASAC-LTR-96-008) 

Summary of CASAC Panel Members Recommendations 

(all units ng/m^) 





PM2.5 
24-hr 


PM2.5 
Annual 


PMio 
24-hr 


PM10 
Annual 


Current NAAQS 


N/A 


N/A 


150 


50 


EPA Staff Recommendation 


18-65 


12.5-20 


150^' 


40-50 




Name 


Discipline 










Ayres 


M.D. 


yes^ 


yes^ 


150 


50 


Hopke 


Atmos. Sci. 


20 - 50' 


20-30 


no 


40 -50' 


Jacobson 


Plant Biologist 


yes' 


yes' 


150 


50 


Koutral<is 


Atmos. Sci. 


yes'^^'^ 


yes^'^*^ 


no 


yes" 


Larntz 


Statistician 


no 


25-30' 


no 


2 
yes 


Legge 


Plant Biologist 


>75 


no 


150 


40-50 


Lippmann 


Health Expert 


20 - 50' 


15-20 


no 


40-50 


IVIauderly 


Toxicologist 


50 


20 


150 


50 


IVIcClellan 


Toxicologist 


no' 


no" 


150 


50 


IVIenzel 


Toxicologist 


no 


no 


150 


50 


IVIiddleton 


Atmos. Sci. 


yes^''^' 


yes^'^ 


150'^^' 


50 


Pierson 


Atmos. Sci. 


yes^^ 


yes^'^ 


4 
yes 


yes"^ 


Price 


Atmos. Sci./ 
State Official 


yes^^^" 


yes^" 


no^'^ 


yes"^ 


Sliy 


Epidemiologist 


20-30 


15-20 


no 


50 


Samef 


Epidemiologist 


yes'^^^ 


no 


150 


yes' 


Seigneur 


Atmos. Sci. 


yes'^^ 


no 


150^' 


50 


Speizer^ 


Epidemiologist 


20-50 


no 


no 


40-50 


Stolwijl< 


Epidemiologist 


75' 


25-30' 


150 


50 


Utell 


M.D. 


>65 


no 


150 


50 


White 


Atmos. Sci. 


no 


20 


150 


50 


Wolff 


Atmos. Sci. 


>75''^ 


no 


150' 


50 



not present at meeting; recommendations based on written comments 

declined to select a value or range 

recommends a more robust 24-hr. form 

prefers a PMi 0.2.5 standard rather than a PMio standard 

concerned upper range is too low based on national PM2.5/PM10 ratio 

leans towards high end of Staff recommended range 

desires equivalent stringency as present PMio standards 

if EPA decides a PM2.5 NAAQS is required, the 24-hr. and annual standards should be 

75 and 25 ng/m', respectively with a robust form 

yes, but decision not based on epidemiological studies 
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^°low end of EPA's proposed range is inappropriate; desires levels selected to include 
areas for which there is broad public and technical agreement that they have PM2,5 
pollution problems 

"only if EPA has confidence that reducing PM2 5 will indeed reduce the components of 



particles responsible for their adverse effects 
'concerned lower end of range is too close to background 

'the annual standard may be sufficient; 24-hr level recommended if 24-hour standard 
retained 
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Roger O. McClellan, DVM, MMS, DSc (Honorary), 

Diplomate-ABT, Diplomate-ABVT, Fellow-ATS 

Advisor, Toxicology and Human Health Risk Analysis 

13701 Quaking Aspen Place N.E. 

Albuquerque, NM 87111-7168 

Tel: 505-296-7083 

E-mail: roger .o.mcclellan @ att.net 

March 18, 2006 



TO: 



Dr. George Gray 

Assistant Administrator for the Office of Research 
and Development 

Bill Wehrum 

Assistant Administrator for the Office of Air and Radiation 



THRU: 

FROM: 

RE: 

Summary 



Vanessa Vu 

Director, Science Advisory Board Staff Office 

Roger O. McClellan ^ ^ 

Comments on National Ambient Air Ouality Standards Review Process 



The process by which National Ambient Air Quality Standards (NAAQS) have 
been developed in accordance with the requirements of the Clean Air Act, including the 
role of the Clean Air Scientific Advisory Committee (CASAC), has evolved over nearly 
three decades. In my opinion, this process is one of the Agency's most successful efforts 
in the use of science to inform public policy decisions, i.e., the setting of NAAQSs for 
criteria pollutants. However, the process can be improved and is deserving of the critical 
review it is currently being given by the Agency's senior management. I urge that all 
elements of the NAAQS setting process be reviewed from (a) the identification of 
research priorities for the Agency's research on criteria pollutants to (b) funding and 
conduct of research through (c) the creation of Criteria Documents, (d) conduct of risk 
assessments, (e) development of Staff Papers to (f) the policy decisions and public 
comment periods that ultimately result in (g) continuation of existing standards or their 
revisions. 

It is clear CASAC has an important and critical role in the process. In my 
opinion, that role is to ensure that all of the available scientific literature and its attendant 
certainties and uncertainties is reviewed, interpreted, synthesized and integrated to inform 
the policy decisions that must ultimately be made by the Administrator. A major 
challenge in the overall process is that scientific information, especially in the life 
sciences, always contains significant biological variability and usually has substantial 
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uncertainties. Moreover, the issues at hand involve human health and welfare issues that 
people highly value. The scientists involved in the NAAQS process, both within the 
Agency and from outside, including CASAC members, are not only highly competent 
scientists, they have personal values that influence their decisions on all matters 
extending from the interpretation of individual papers to the range of numerical levels 
considered for setting a NAAQS. In my opinion, the major challenge to improving the 
NAAQS setting process is to have as complete an exposition of the scientific information 
relevant to setting the Standard as possible with an even-handed consideration of 
uncertainty without infusion of personal values. Value considerations should only be 
considered in the policy decision. 

In my opinion, the Criteria Document development process needs to be 
streamlined with more attention given to evaluation, synthesis and integration of 
information relevant to decisions on the four elements of the NAAQS; (a) indicator, (b) 
averaging time, (c) numerical level, and (d) statistical form. More concisely written 
criteria documents that relate the critical information, including even-handed exposition 
of uncertainties, will allow the CASAC to focus on these critical components and avoid 
becoming bogged down in editorial detail and review of material not relevant to setting of 
the NAAQS. Substantially more attention needs to be given to developing risk 
assessments that more adequately consider alternative exposure-response models and 
consider the full range of uncertainty in the underlying scientific information. More 
adequate risk assessments will also consider confounders that impact on the estimated 
risks of a particular pollutant and also serve as a benchmark for the reality of the risk 
assessment process. 

Better risk assessments, carried out by multiple organizations, will lead to 
improved Staff Papers. When presented with improved Risk Assessments and Staff 
Papers, CASAC can first focus on the quality of the analyses irrespective of the specific 
outcome of the analyses. I applaud the presentation in the Staff Papers of relatively broad 
ranges for standard setting. I view the ranges as reflecting scientific uncertainty. Indeed, 
I urge that in future documents the range not necessarily be anchored at the upper bound 
by the current standard. This is the case because the previous NAAQS was set based on 
both science and policy considerations. The ranges in the Staff Paper should be science- 
based and reflect any uncertainties in the science. The CASAC review of the Staff 
Papers should focus on the scientific content and avoid the temptation to introduce 
personal policy preferences in the range endorsed by CASAC. In my opinion, CASAC 
needs to avoid endorsing narrowly defined ranges, such as it recently did in suggesting a 
range of 13-14 |ag/m^ for the annual PM2.5 standard. In my view, this places CASAC in 
the role of setting the standard and ties the hands of the Administrator in considering 
policy options. In my opinion, the substantial quantitative uncertainties in PM2.5 
exposure-response relationships speak against CASAC proposing such a narrow range. 

Recent efforts by the Science Advisory Board staff to relegate CASAC to a role 
similar to other SAB Committees have been totally inappropriate and requires critical 
review and reversal. CASAC is a scientific advisory committee charged with 
responsibility for advising on matters of extraordinary national importance. All of 
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CASAC deliberations and its conclusions need to be reached in public sessions. The 
current excessive use of teleconferences without attendant transcripts should be reduced. 
There is a need for CASAC to provide much more succinct, science-based letters to the 
Administrator including the use of "closure letters" at critical junctures in the process. 
The objective should be to ensure that the science as presented in the Staff Paper is 
appropriately documented to inform policy decisions. If necessary, this may require that 
CASAC representatives participate in dialogue with the Courts to avoid the imposition of 
Court-imposed time schedules that may not allow adequate time for the preparation and 
CASAC s iterative review required to ensure scientific quality. CASAC should avoid the 
temptation of creating in its letters an alternative to the Criteria Documents or Staff 
Papers, the focus should be on achieving scientifically acceptable EPA documents. In 
commenting on contentious issues, CASAC should also avoid striving for a single 
consensus view when there are divergent science-based opinions within the CASAC 
Panel. 

There is a critical need for the Executive Branch and the Congress to work 
together to amend the Clean Air Act to increase the review cycle from 5 years to 10 
years. The 5-year cycle is no longer appropriate and needs to be lengthened to be better 
matched to how new information is acquired and to allow for an improved process of 
evaluation, synthesis and integration of information prior to presentation of documents to 
the CASAC. 

COMMENTS 

These comments have been prepared in response to your request of February 17, 
2006 for input to your review of the process used to periodically review and revise, as 
appropriate, the air quality criteria and National Ambient Air Quality Standards 
(NAAQS) as required by Sections 108 and 109 of the Clean Air Act. My comments 
consist of a background section, a section on the operation of the Clean Air Scientific 
Advisory Committee and from sections that address the key questions you have posed. 

Professional Background 

I will briefly review my personal involvement with CASAC as background for 
consideration of my comments. My professional experience has been in the fields of 
inhalation toxicology, aerosol science and risk analysis. I have also had extensive 
experience serving on numerous U.S.E.P.A. Advisory Committees beginning with 
service on the Agency's first Science Advisory Board and continuing today with service 
on the current CASAC PM Panel. This included chairing an ad hoc Committee to review 
the Agency's first Criteria Document on Airborne Lead, service on the first EPA review 
panel for particulate matter, service on numerous CASAC panels that have considered all 
of the Criteria Pollutants, service as a CASAC member including four years (1988-1992) 
as Chair of CASAC, service on the CASAC PM and Ozone panels that advised on the 
last round of standard setting and service on the current CASAC PM Panel. 
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In addition to my EPA service, I have served on numerous other advisory 
committees in both the public and private sectors that have been concerned with air 
quality issues. This has included the National Academy of Sciences/National Research 
Council Committee (NAS/NRC) that prepared the report - "Science and Judgment in 
Risk Assessment" and the more recent NAS/NRC "Committee on Research Priorities for 
Airborne Particulate Matter" that prepared four reports (1998, 1999, 2000 and 2004). 

Personal Reflections on CASAC Modus Operandi 

Prior to the creation of the EPA in 1970, responsibility for administering the 
Clean Air Act and earlier air quality statutes was vested with the National Air Pollution 
Control Administration which had an independent Clean Air Advisory Committee. With 
creation of the EPA, a number of "inherited" advisory committees, including the Clean 
Air Committee, were abandoned. In their place the EPA created a Science Advisory 
Board which had a number of discipline- oriented committees; Health, Engineering, 
Ecology, etc. I served as a member of the original EPA Science Advisory Board 
Executive Committee by virtue of my chairing the Board's only original issue-oriented 
standing committee, the Environmental Radiation Exposure Advisory Committee. 

In the early 1970s, air quality issues were handled by the SAB on an ad hoc basis. 
An example was the handling of a review of lead as an air pollutant. Lead had not been 
included as one of the original criteria air pollutants. The National Resources Defense 
Council (NRDC) took legal action to have lead listed as a criteria pollutant and ultimately 
prevailed in the Appeals Court (NRDC vs Train). Thus, EPA was required to prepare a 
criteria document on airborne lead and the decision was made to have the document 
subjected to external peer review. Since a formal clean air scientific review committee 
did not exist, I was asked to chair an ad hoc committee to review the lead criteria 
document. The ad hoc committee met in public sessions, reviewed the report, received 
input from the EPA staff and heard public comments. 

The Committee's initial conclusion was that the original criteria document on 
airborne lead was inadequate and needed to be substantially revised. EPA was operating 
under a court-ordered deadhne to issue a NAAQS for lead, a deadline that did not allow 
adequate time for revision of the lead criteria document. However, the Agency and 
interested parties persuaded the Court to extend the deadline to allow preparation of a 
scientifically adequate document rather than merely meeting an arbitrary "date certain" 
deadline. The ad hoc committee reviewed subsequent revisions of the document. 
Ultimately, a document was created that the ad hoc committee approved as being a 
scientifically adequate basis for setting the National Ambient Air Quality Standards for 
lead and issued a "closure letter" to the EPA Administrator. The key points being made 
are that the scientific basis for the NAAQS for lead was reviewed and a decision was 
made by the ad hoc committee as to when the documentation was scientifically adequate 
for regulatory decision making. 

In my opinion, the approach taken by the ad hoc committee dealing with lead as a 
criteria pollutant influenced the decision of the Congress in amending the Clean Air Act 
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in 1977 to explicitly call for the creation of an independent Clean Air Scientific Advisory 
Committee (CASAC). The CASAC, in accordance with the Clean Air Act (1977), has 
periodically reviewed the scientific basis for setting and revising the NAAQS for all the 
criteria pollutants. I have participated in most of those reviews and served as Chair of 
CASAC (1988-1992). CASAC has reviewed all of the Criteria Documents for criteria air 
pollutants prepared by EPA's Office of Research and Development and in some cases, 
health assessment documents for specific pollutants, such as diesel exhaust. In every 
instance, the CASAC modus operandi has included rigorous review of the document, 
receipt of input from the EPA staff and receipt of extensive written and oral comments 
from interested parties. Until recently, all of these activities were carried out in public 
face-to-face sessions. On many occasions, the CASAC has offered comments to the 
Agency on iterative draft documents and, when it deemed the documentation 
scientifically adequate for regulatory decision making, provided a "closure letter" to the 
EPA Administrator. Without question, the CASAC has played a critical role in ensuring 
that the "final" criteria documents were of high scientific quality. 

As the Criteria Documents grew in size the CASAC recognized the value of 
having documentation that could bridge from the science of the criteria document to the 
regulatory decision-making process. This was the genesis of the "Staff Papers" prepared 
by EPA's Office of Air Quality Planning and Standards. The CASAC reviewed the Staff 
Papers, received EPA input, received public comments and deliberated in public sessions 
on the scientific adequacy of the documentation. Frequently, the CASAC advised the 
Agency that the current version of the Staff Paper was not scientifically adequate and 
needed to be revised. In a manner similar to that followed with the Criteria Documents 
the CASAC provided a "closure letter" on the Staff Paper to the EPA Administrator when 
it deemed the Staff Paper scientifically adequate for regulatory decision making. The 
"closure letters" on the Staff Papers have typically included comments on the proposed 
range for setting the NAAQS. 

The discussion here is not intended to be an exhaustive review of all of CASAC s 
activities; rather the review has focused on the modus operandi of CASAC as a standing 
independent scientific committee. The activities of the CASAC, in my opinion, have 
been in accord with the language and intent of the Clean Air Act (1977) and consistent 
over time with the evolution of CASAC practices that have received substantial public 
and legal scrutiny. The modus operandi has proved successful in helping to ensure that 
the NAAQSs are science-based. 

It now appears that parties within the EPA, but unknown to the public, have 
changed the modus operandi of the CASAC. The arguments for change have been made 
in "administrative sessions" of the CASAC and, thus, have not been made public. As 
best I can discern the changes are intended to relegate the CASAC to a status similar to 
other Committees under the SAB umbrella operating under the Federal Advisory 
Committee Act (FACA) rules. The motivation for the changes has not been publicly 
articulated. Does the Agency believe that its ability to carry out the mandates of the 
Clean Air Act have been impaired by previous rigorous CASAC review and the use of a 
"closure letter" process? If so, this should be publicly documented. I would argue that to 
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the contrary, even the delays resulting from CASAC's call for more rigorous 
documentation of the science have contributed to more defensible NAAQSs. 

Is the argument one that the CASAC is operating in a manner that is different 
from some other EPA FACA committees? If so, then the differences need to be publicly 
documented. Even if differences do exist in how CASAC operates versus other FACA 
committees that does not make the CASAC past modus operandi inappropriate. The 
critical issue is whether the CASAC has and is operating in a manner consistent with the 
Clean Air Act language calling for an independent CASAC and FACA. Over the past 25 
years, numerous Chairpersons and members of CASAC have appeared before 
Congressional Committees. My impression is that the Congress has consistently held a 
favorable view of the CASAC's modus operandi and its role in implementing the Clean 
Air Act. I am not aware that either the Congress or senior members of the Executive 
Branch have advocated changes in how CASAC carries out its responsibilities. 

Timeliness of the NAAQS Review Process 

What are your views on the timeliness and efficiency of the current process for 
both EPA 's and CASAC's reviews of the air quality criteria and the NAAQS, in terms of 
the time that is spent between the start of the review and the publication of the Agency's 
proposed decisions on the standards? 

Can you identify structural changes to the process and/or key documents (e.g., the 
Criteria Document, Staff Paper, Risk Assessment) or changes in the Agency 's 
management of the process that could shorten this timeframe while preserving an 
appropriately comprehensive, transparent and policy-relevant review and allowing 
adequate opportunities for CASAC review and advice and for public comment on these 
documents? 

In my opinion, the CASAC and its numerous Review Panels have generally 
participated in the NAAQS review process in a timely and efficient manner. Indeed, I 
think the CASAC Review Panels have, on some occasions, been excessively concerned 
with meeting court-imposed time schedules rather than focusing on the scientific quality 
of the end product. In my opinion, that occurred with the current Particulate Matter 
review when serious statistical issues arose delaying the finalization of the Criteria 
Document. The issue was further confounded by the Agency prematurely releasing a 
draft Staff Paper. I note it was released prematurely because the Criteria Document was 
not yet finalized. A draft Staff Paper should never be released to CASAC and the pubhc 
prior to the Criteria Document being finalized. Because of the Court-ordered schedule, 
the CASAC Review Panel did not have time for an iterative in-depth review of the Staff 
Paper. Indeed, the CASAC PM Panel offered comments on key aspects, namely the 
PMio-2.5 indicator, of the Staff Paper after it was finalized in June 2005. As a result, the 
Staff Paper did not meet the scientific quality standards I expect. In my opinion, this 
contributed to what I view as a needless debate over whether the Annual PM2.5 Standard 
proposed by the Administrator was scientifically out of bounds as charged by some 
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members of the CASAC PM Panel. They argued that the Panel had recommended that 
the Annual PM2.5 standard be set in the range of 13 to 14 |ag/m^ . 

The Staff Paper related an upper bound of 15 |ag/m^ which was selected by the 
Administrator. My personal opinion is that 14 |ag/m^ and 15 lag/m"* are both consistent 
with the available scientific evidence and the substantial quantitative uncertainty in the 
health benefits at these ambient concentrations. The bottom line is that the NAAQS 
review process schedule should be driven by concern for scientific quality and not by 
court-imposed deadlines. My personal experience is that the Courts will yield to a 
scientific quality standard if appropriate progress is being made. With regard to the 
recent round of the Particulate Matter review, I am at a loss as to why EPA attorneys 
were not willing to argue for scientific quality of the products as trumping meeting court- 
imposed deadlines that were unrealistic. 

A major factor in the timeliness of the NAAQS review process relates to the 
timeliness of the EPA staff preparation of the (a) Criteria Document, (b) Risk 
Assessments, and (c) Staff Papers. The time required for their development is dependent 
upon the efficiency and knowledge of the staff and the resources available. In my 
opinion, the Criteria Documents could be substantially reduced in scope and size if the 
documents were to focus on the knowledge base used for decisions on the four elements 
of the NAAQS; (a) indicator, (b) averaging time, (c) numerical level, and (d) statistical 
form. There is no need to create an encyclopedia covering everything known on each 
criteria pollutant. Moreover, I suspect that at least some of the EPA contractors drafting 
chapters for the Criteria Documents do not really understand the need to focus on the four 
elements of NAAQS. However, they are not alone; many new CASAC Panel members 
are slow to grasp this concept. 

The efficiency with which Criteria Documents and Staff Papers are developed 
could be substantially enhanced if the process were to incorporate modern informatic 
tools and processes. For example, it would be helpful if every piece of literature 
considered for potential citation in the Criteria Documents should be available in an 
electronic file searchable by EPA staff, authors, CASAC Panel members and the public. 

Increasingly, the contents and quality of Staff Papers are built on formal 
quantitative risk assessments. The development of these risk assessments is a weak link 
in the overall NAAQS review process. I find it disappointing that the EPA staff are 
apparently not capable of carrying out these crucial analyses. I say "apparently 
incapable" because the assessments are actually conducted by EPA contractors. I am 
concerned that so much dependence is placed on one risk assessment performed by a 
single contractor. In my view, the risk assessments are not state-of-the-art and tend to 
over-emphasize scientific certainty and under-state the substantial scientific uncertainty 
present in the quantitative estimates of health risks for current air quality and the 
projected health benefits of various potential standards. A big step forward would occur 
if multiple parties, including the EPA staff, were to develop risk assessments. I am not at 
all concerned about having "dueling" risk assessment results. It would be refreshing to 
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have the opportunity to see how different parties use and model the available scientific 
data. 

The last step in the process bridging from the science on a particular criteria 
pollutant to the NAAQS is the Staff Paper. As noted earlier, draft Staff Papers should not 
be released until after the Criteria Document is finalized and the risk assessment is 
available. The premature release of the PM Staff Paper, based on a Criteria Document 
that was still being revised and did not adequately document uncertainty, led to an 
extended period of time for the various special interests to advocate for specific final PM 
standards. Unfortunately, the authors of the Staff Paper are limited in their ability to 
characterize the certainty/uncertainty in the science when only a single and, perhaps, 
flawed risk assessment is available. I have been pleased that Staff Papers have wisely 
identified relatively broad ranges for potentially setting standards. For example, I think 
the staff's use of a range of 12-15 |Jg/m^ for the Annual PM2.5 standard reflected their 
views of the uncertainty in the underlying science. I think the CASAC PM Panel 
narrowing the range to 13 to 14 |ag/m^ reflected some CASAC members lack of 
appreciation of the scientific uncertainties. Other individuals argued for consideration of 
the margin of safety, which is a policy consideration for the Administrator, and reducing 
the upper bound of the range to 14 |Jg/m^ . Alternatively, some of the Panel members 
perhaps wanted to make a "policy statement" that they wanted the current PM2.5 standard 
reduced below 15 |ag/m\ 

As I close this section, I want to enter a plea for transparency and public 
deliberation on the contents of the Criteria Documents, Risk Assessments, and Staff 
Papers. I think it was a travesty that the only face-to-face meeting on the last draft PM 
Staff Paper was only a day and a half in length complemented by brief teleconferences. 
Moreover, a transcript does not exist for many of these "public meetings." The result is 
that little deliberation occurred in public. Fortunately, Panel members were given the 
opportunity to append their individual comments to the CASAC PM Panel letter. I urge 
you to carefully review the transcript of the April meeting and the CASAC letter and 
individual comments to gain an appreciation of the extent to which the scientific 
uncertainties were considered. 

More recently, the CASAC PM Panel held a brief teleconference to discuss the 
Agency's Proposed Rule. The meeting involved very limited deliberation, to the extent 
20 individuals can deliberate on a teleconference and reached few firm conclusions. 
Moreover, a decision was initially made to not append comments of individual members 
to the CASAC PM Panel letter to the Administrator. The stated reason, off the record, 
was a desire to present the Administrator with a clear consensus letter. In my view, this 
approach does not serve the Administrator, the Agency, or the scientific community well. 
By suppressing divergent views an artificial sense of scientific certainty is conveyed. 

The last comment I make on the timeliness of the NAAQS review process is the 
desirability of the Agency working with Congress to amend the Clean Air Act to extend 
the review cycle from 5 years to 10 years. Five years is a short period of time in the 
world of scientific research. A review every 10 years would provide more adequate time 
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to develop new information and to interpret, synthesize and integrate it for use in the 
NAAQS setting process. Moreover, extending the review cycle to 10 years would allow 
for more realistic planning and conduct of the kind of research that has the greatest 
impact on revision of the NAAQS. In my view, the kind of research conducted in the 
past has been inappropriately truncated to fit the 5-year review cycle. 

Consideration of the Most Recent Available Science 

To enhance the Agency 's ability to take the best and most recent available science 
into account in making decisions on the standards, can you suggest changes in the 
process and/or key documents that could shorten the time between the presumptive cutojf 
date for scientific studies evaluated in the review and reaching proposed decisions on the 
standards, or that could otherwise facilitate appropriate consideration of more recent 
studies ? 

I take exception to the use of the evaluative word, best, in the question - "To 
enhance the Agency's ability to take best and most recent available science into account - 
-." Best is an evaluative word; what is best to me may not be best to another scientist. 
In my opinion, during the development of the Criteria Documents all of the available 
published literature available by a presumptive cutoff date should be evaluated and 
considered for inclusion in the Criteria Document. I suggest that what is included in the 
Criteria Document should be those published papers that have bearing on the four 
elements of the NAAQS, namely, (a) the indicator, (b) the averaging time, (c) the 
numerical level, and (d) the statistical forms. I recognize that in taking this approach, a 
substantial number of papers, including some published by CASAC Panel Members, may 
not appear in the Criteria Document. The list of evaluated, but not cited, papers could be 
included as an Appendix or Supplement to the Criteria Document. 

If the approach I have suggested is taken, the Criteria Document will be shortened 
and the interval between the cut-off date and presentation of the document to the CASAC 
Panel will be shortened. If the EPA staff were to routinely evaluate papers as they are 
published rather than waiting until an external contractor prepares a prospective chapter, 
the papers could be more readily integrated into the Criteria Document. I emphasize the 
need for evaluation, integration and synthesis of a body of literature because that should 
be the intent of the Criteria Document. It should not be a mere recitation of what is found 
in a series of papers. This integration and synthesis function is an area where the staff 
preparing the Criteria Document frequently falls short. All too often they depend on the 
CASAC Panel to carry out this function. If the EPA staff were to provide a Criteria 
Document with improved integration and synthesis of the available relevant literature, the 
CASAC Panels could focus on critical issues. For example, in my view the CASAC PM 
Panel spent much too little time in public sessions deliberating on the nature of the 
exposure-response function for PM2.5 and PMio-2.5 and the associated uncertainties. 

In my opinion, the Criteria Document did not adequately explore the difficulty, if 
not the impossibility, of demonstrating the presence or absence of thresholds. Even less 
attention was given to exploring the potential range of exposure-response models and the 



Case 1:05-cv-01814-LFO Document 23-5 Filed 05/31/2006 Page 45 of 77 



validity of using a single linear exposure-response function down to levels approaching 
background. Quite simply, the recent PM Criteria Document overstated the degree of 
scientific certainty of knowledge on these matters. This continued in the Risk 
Assessment and Staff Paper. The result was a presentation of results expressed as "body 
counts" at levels in the range of the current NAAQS that understated the substantial 
uncertainty that I viewed as being present. I ask - is it reasonable to conduct a risk 
assessment for different regions of the United States and with PM of markedly different 
composition and population with markedly different patterns of disease and, basically, 
using a single exposure-response coefficient? I think this approach is a misuse of science 
to achieve a pre-determined objective held by some individuals - a reduction in both the 
annual and 24-hour PM2.5 standard. 

Distinction Between Science and Policy Judgments 

Recognizing that decisions on the standards, while based on the available 
science, also require policy judgments by the Administrator, what are your views on how 
clearly scientific information, conclusions, and advice are distinguished from policy 
judgments and policy recommendations on the standards throughout the review process? 

Can you suggest changes in the process and/or changes to the format and 
contents of key documents that would help to make these distinctions clearer? 

The issue of separating scientific evaluations from policy decisions is vexing and 
is increasingly an issue in the functioning of the CASAC Panels. It would be helpful if, 
at each step in the NAAQS process including each meeting of the scientists preparing the 
Criteria Documents and the Staff Paper and their review by CASAC, if each participant 
were reminded - "Every individual should recognize the distinction between scientific 
evaluation and policy decisions and recognize that the matters being dealt with are at the 
interface of science and policy. Each individual participant is asked to leave their 
individual ideologies and thoughts on policy decision outcomes at the door before 
deliberating on the science. " This is not a matter of an individual's employment, i.e., 
academic, government, industry, etc. or political affiliation. It applies to all participants. 
This is an especially vexing issue for scientists involved in evaluating their own research 
results or that of close colleagues. In today's resource constrained world everyone wants 
to have their work used in the public arena, moreover, they would like to see the door left 
open or opened wider for them to do more work on the topic under consideration. 
Indeed, some individuals, including CASAC Panel Members, desire a "sense of 
accomplishment" - some individuals interpret that as - did we participate in lowering the 
NAAQSs? Some have suggested that there would be a "limited sense of 
accomplishment" if only the 24-hour PM2.5 standard were lowered and the Annual PMi.s 
standard was left unchanged. Yes, scientific evaluations and policy decisions do get 
inter-twined by individual scientists in expressing their own personal preferences on life 
science issues. 
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Repeatedly, one hears the view expressed that is necessary to be "protective of 
pubUc health." I agree, however, I think that precautionary considerations are part of the 
policy decision, not the interpretation and integration of the science. 

I think the NAAQS setting process can be improved if participants (and this 
includes EPA staff, EPA contractors and CASAC Panel members) are continually 
reminded of the need for distinguishing between scientific evaluation and policy 
decisions. This is not an end of NAAQS process issue. It needs to start with evaluation 
of the published literature and carry through to the final rule making. It is critical that at 
each step all of the uncertainties be exposed. It is not appropriate, as happens all too 
often, to argue that this is a human health issue and it is necessary to be conservative. 
There is a need to relate all of the scientific uncertainties and then let the degree of 
conservatism be addressed in a Policy decision. Scientists are reluctant to take that 
approach because they generally have a deep-seated mistrust of the individuals making 
the policy decisions. This issue is becoming increasingly important in dealing with the 
NAAQS for criteria pollutants and the challenge of deciding "how low is low enough." 
This was exemplified by the recent "fire storm" over the issue of the Administrator 
proposing to continue the Annual PM2.5 standard at 15 |Jg/m^ and many on the CASAC 
PM Panel who argued it should be set no higher than 14 |Jg/m^ . I personally know of no 
scientific information or scientific methodology that would conclude 14 |ag/m^ is 
scientifically acceptable and 15 |ag/m^ is scientifically unacceptable. 

Identifying, Characterizing, Quantifying and Communicating Uncertainties in 
Scientific Information 

Recognizing the importance of characterizing and clearly communicating the 
uncertainties in the science and quantifying uncertainties in exposure and risk estimates 
as explicitly as possible, what are your views on any changes in the process and/or 
changes to the format and content of key documents that might facilitate a more 
complete, quantitative, and policy -relevant characterization of uncertainties? 

I have already opened the discussion on scientific uncertainties. This is probably 
the weakest aspect of the total NAAQS process and presents, albeit controversial, the 
greatest opportunities for improvements in the process. Unfortunately, this is a long and 
steep slope to ascend. The roots of the issue begin with the funding, planning, conduct 
and reporting of research. Research gets funded on the "sky is falling" issues and on the 
perpetuation of the "sky is falling" issues. Only limited attention is given to developing 
"issue-resolving" research strategies. Moreover, when the research is reported, a "the sky 
is falling" paper may consider some risk factors like PM or O3 in isolation to magnify 
their importance. 

Indeed, many epidemiological papers fail to adequately consider other potentially 
serious confounders such as cigarette smoking and other air pollutants and local and 
regional demographic differences. Many of the laboratory studies using isolated cell 
systems or laboratory animals use single exposure/dose levels and one or a very few 
short-term observation periods. The emphasis, time and time again, is on demonstrating 
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hazards that can be automatically equated to occurring in humans. In short, few research 
investigations are planned and conducted with a view to quantitatively assessing human 
risk. All too often, it is not appreciated that at the end of the NAAQS process, it is 
necessary to be quantitative. Inevitably, in the NAAQS process papers that yield 
negative outcomes are given scant attention. The view frequently expressed is that "you 
know you cannot prove a negative, the study design must be flawed." The evaluation 
"playing field" is clearly not level. Negative findings need to be more adequately 
considered when relating the scientific uncertainties that under-gird the NAAQS. 

In my opinion, the development and presentation of comparative risk data, 
especially when it is for the same population studied to evaluate a pollutant effect, is 
extremely valuable for policy-decision making. I recognize that the Administrator is 
forced by the Clean Air Act to consider each Criteria Pollutant on a pollutant by pollutant 
basis. Nonetheless, in making a policy decision on "how low is low enough," I think the 
Administrator would be well served by knowing what were the quantitative estimates for 
excess risk for cigarette smoke and other pollutants for the same disease outcomes in 
making policy decisions on a given criteria pollutant. In some cases, the exposition of 
such data can prove very insightful. 

The final steps in the NAAQS process where uncertainties are dealt with for 
communication to the Administrator are in the Risk Assessment and Staff Paper. Both 
the Risk Assessment and Staff Paper do a poor job of acknowledging and characterizing 
scientific uncertainties. Because these documents are linked, the difficulty starts with the 
Risk Assessment. This is especially unfortunate since some individuals look at any 
number (as reflected in body counts) as being highly precise and certain. In my view, 
they are usually very uncertain. 

The NAAQS process could be improved by opening up the risk assessment 
process. The present system typically uses a single EPA contractor organization which 
may well have an excessively close relationship to the EPA staff. The EPA staff are 
experts in this arena. Why not have them conduct risk assessments? This might well be 
complemented by having several other organizations preparing risk assessments. Such a 
process might illuminate some of the uncertainties and how they influence the 
quantitative estimates of risk and the attendant bounds. I would be pleased to have 
"dueling risk assessments" if they helped reveal the underlying scientific uncertainties. 

With regard to characterizing uncertainties, I think it appropriate to raise the issue 
of eliciting "expert judgments." I have participated in these processes working with EPA 
contractors, so I can speak from experience. In my experience, the "playing field" does 
not start level. It would appear that the EPA exerts a strong influence in introducing the 
key studies to be considered in the process and in selecting the participants. In the 
process I participated in there was major emphasis given to attempting to elicit linear 
exposure-risk coefficients. I was disappointed to learn that the process I participated in 
has been extended by EPA staff to a larger group of participants. One could argue that 
the participants have been overly selected by EPA, perhaps to help ensure the answer is 
that which is desired by EPA staff. Again, the emphasis is apparently on arriving at a 
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single linear exposure-response coefficient. To help ensure agreement, I understand the 
process is designed to be iterative with the group re-assembled to review and refine the 
first round outcome. I am of the opinion that it would be useful to have a much larger 
and more diverse group of expert participants involved and place the emphasis on 
eliciting not only a "central estimate," but the range of uncertainty in the opinions. Of 
even greater importance, it would be useful for EPA to support a much broader program 
of research on alternative methods for evaluating exposure-response relationships for 
extraordinarily low levels of excess risk attributed to air pollutants. 

Selection of CASAC Members and CASAC Panel Members 

Beyond the questions asked, I think it is important to give some attention to how 
CASAC members and CASAC Panel members are identified and selected. I want to start 
the discussion by relating my personal high regard for each individual I have served with 
on CASAC Panels - it probably is well in excess of 100 individuals. I will also relate the 
view that service on CASAC and the CASAC Panels is demanding and has few rewards. 
However, I am concerned that member selection deserves careful review and scrutiny. 

The present process is alleged to be open and transparent. While the "public 
calls" for candidates via the Federal Register and calls to professional organizations have 
appeal, it also has shortcomings. Increasingly, professional organizations have taken an 
advocacy role by publishing in Professional Journals their views on desired outcomes for 
NAAQS, i.e., an Annual PM2.5 standard of 12 |Jg/m\ I belong to at least one of these 
organizations and view it as being a highly meritorious Professional Society. However, I 
am concerned that in advancing its views, it is expressing a view based on both science 
and policy considerations. It is appropriate to ask on what basis such organizations 
nominate candidates for CASAC Panels? In the nomination process there is a perception 
that any industry association is a "kiss of death" for participation on a CASAC Panel or, 
indeed, SAB Committees concerned with hazard/risk issues. 

My concern for individual scientists being able to disentangle science and policy 
issues covers the landscape - academic, government, industry and non-government 
organization employees. Few individuals do it well and many cannot do it at all - that is 
human nature. I also have concern for the role of the EPA staff in nominating individuals 
or encouraging the nomination and selection of individual members of CASAC or 
CASAC Panels. Part of the concern for the role of EPA staff relates to the perception in 
some quarters that the EPA staff, in general, is quite "risk averse." I am also concerned 
at the use of a two-tired system; one for selecting CASAC members allegedly appointed 
by the Administrator and consultants appointed by the SAB office. It would appear that 
the former appointments receive more scrutiny than the latter. I suggest both CASAC 
members and consultant appointments deserve the same degree of scrutiny. The process 
by which CASAC members and consultants are selected deserves careful review. 
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CASAC Processes 

The CASAC occupies a relatively unique role within the EPA. It is a 
Congressional mandated independent scientific committee, however, it is housed within 
and managed by the Office of the Science Advisory Board. I have very serious concerns 
about how the CASAC has been managed by the SAB in recent years. There appears to 
have been an effort to "stove pipe" the CASAC into the same mode of operation as other 
SAB committees. The SAB staff supporting CASAC appear to give little consideration 
to the historical context for CASAC s operation. Major attention has appropriately been 
given to creating a CASAC that functions flawlessly and seamlessly in accordance with 
all the applicable statutes. 

In my opinion, little attention appears to be given to creating and managing a 
process that values scientific discussion and deliberation. Major process issues have been 
announced in "executive sessions" without any explanation to the public. When 
questioned, the answer on some occasions is the lawyers recommended the change, for 
example, abandoning the "closure letter" process in favor of offering "advisory letters" 
and issuing letters from CASAC despite the fact that the matter in question was 
considered by an entire CASAC Panel. Irrespective of the basis for the decisions, since 
they relate to a public science advisory committee, they should be announced and 
explained in public sessions. 

The SAB Office has apparently abandoned the process of creating "transcripts" 
for all CASAC meetings. Transcripts are needed for all meetings, irrespective of whether 
they are face-to-face meetings or teleconferences. This is unfortunate since the 
proceedings of all CASAC Panel meetings should be available for public review. Indeed, 
as a past CASAC Chair, I can relate that I found such transcripts very useful in recalling 
what occurred at a given meeting and in the development of letters to the Administrator. 
The use of teleconferences has become a part of the modus operandi for CASAC Panels. 
Teleconferences may be useful for conveying information; they are not effective for 
scientific discussion and deliberation among a Panel of 20 members. For example, I am 
at a loss as to why a teleconference was used for discussing and reaching a CASAC 
position on matters of such importance as (a) the PM 10-2.5 Indicator or (b) the CASAC 
PM Panels views on the Proposed PM Rule. If these matters are of substantial national 
importance, and I think they were and are, then in my opinion they deserved a face-to- 
face meeting of the CASAC PM Panel. On these, as well as other CASAC matters, I 
have heard budget constraints advanced as dictating the actions. I find this a "hollow 
argument" for an Agency with a budget in excess of $8 billion per year. If the work of 
CASAC is of vital national importance, and I think it is, then I think the Administrator 
and staff need to find the funds to have CASAC do its work in the appropriate manner. 
That has not been happening. 
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Attachment 3-C 

Additional CASAC Member Individual Comments on EPA's NAAQS Process 



Dr. James Crapo 
March 20, 2006 

The following are my thoughts about how the NAAQS process could be improved: 

Timeliness and Efficiency of the Current Process for both EPA's and CASAC 's Review of the 
Air Quality Criteria and the NAAQS 

Having served on CASAC during the recent and ongoing reviews of ozone, PM, and lead, it is 
my observation that the current process has evolved into an inefficient and ineffective process. 
This leads to major delays and reduces the ability of CASAC to provide rigorous scientific input 
into EPA staff recommendations and policy decisions. The most critical discussions of each 
topic are often delayed until the process is under court order to proceed and adequate time is not 
available for effective and thoughtful interactions. 

The current process requires that a first and second draft of the air quality criteria document 
(AQCD) be prepared [by NCEA-RTP for a given criteria air pollutant] and discussed by 
CASAC, which is then followed by a first and second draft of staff papers which are discussed 
by CASAC, leading to a final CASAC letter to the Administrator. This process produces 
expansive review documents on the literature underlying each subject area but adds years to the 
review cycle and inhibits effective discussion of the critical issue, i.e., whether or not there are 
adverse health effects at current air quality standards. The AQCDs contain no conclusions 
regarding the air quality standard and CASAC is inhibited from meaningfully discussing this 
issue during the process of reviewing the AQCD. The majority of CASAC time is often spent on 
reviewing literature rather than discussing the critical issue of whether or not adverse health 
effects exist at current air quality standards. Finally, the majority of the literature discussed in 
the AQCD focuses on levels that are not relevant to current environmental conditions in the 
United States or the air quality standards. In each review cycle there are only a small number of 
critical scientific studies that address the form and standard of the current NAAQS. These 
critical articles are nearly lost in the massive size of the current AQCD and the process 
established for its review. 

The current review of the air quality standard for lead illustrates the above problems. The 
current air quality standard for lead was set in 1978 and EPA has not conducted a review of this 
standard in the past 15 years. The World Health Organization has set an air quality standard for 
lead that is 3 times lower than the current U.S. standard. This was done in the 1980s. The most 
recent AQCD for lead is a massive document requiring enormous time by the EPA staff to 
prepare and which is still not comprehensive. In addition, after weeks of review and a two-day 
meeting discussing this document, CASAC has not yet discussed the question of whether or not 
the current NAAQS for lead is adequately protective of human health. I would conclude that the 
current process does not allow the EPA and its [Clean Air] Scientific Advisory Committee to 
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effectively address their charge to carry out a timely and effective review of air quality standards 
for the United States. 

I would recommend that the entire process be changed along the following guidelines: 

- Requirement for a comprehensive AQCD should be eliminated. 

- A short AQCD (page-limited) focused only on scientific studies that address relevant air 
pollutant levels in the United States and which address adverse health effects at those levels 
should be prepared. 

- The staff papers should be incorporated into the AQCD reducing this to one document which 
could undergo two or at most three reviews. 

- The air quality document should begin with an interpretation (staff recommendation) on the 
quality of current science relative to the question of adverse health effects at the existing air 
quality standard. The document should then defend that staff interpretation of the scientific 
literature through its summary of studies that directly address the critical question. 

- Comprehensive summaries of the literature should be placed in an appendix and only articles 
deemed to be relevant to the question of the current form and standard for each regulated air 
pollutant should be included in the primary document. 

- The discussions at CASAC meeting should focus on whether or not the current air quality 
standard is adequately protective of human health. 

The above recommended process would be far more efficient in both the use of EPA staff time 
and the time of CASAC members. It would dramatically reduce current inefficiencies and allow 
the EPA to meet its obligation for a timely review of air quality criteria and NAAQS. It would 
also allow CASAC to have a more effective role in defining the scientific basis for changing or 
not changing current NAAQS. 

Consideration of the Most Recent Available Science 

The above recommended change in the process for EPA and CASAC review of air quality 
criteria and NAAQS will substantially enhance our ability to consider the most recent available 
science. No cut-off for a published article to be discussed in the preparation of the final 
document would need to be imposed. Because critical articles relevant to the final decision 
would be considered up to and including the final draft of the document, there would be 
opportunity for them to be considered by EPA staff and discussed by CASAC. By restricting the 
focus to only articles that directly deal with the question of current air quality levels and the 
presence or absence of adverse health effects at current air quality standards, there would be no 
need to impose a cut-off for consideration of best science. 

In summary, the EPA NAAQS review process and policy judgments can be made far more 
efficient if changes are made to require preparation of smaller documents that focus on the 
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question of the adequacy of current air quality standards to protect human health and to allow 
both staff and CASAC to focus on this question from the inception of each review cycle. The 
process could ideally be completed in less than one year, requiring two and at most three cycles 
of discussions with CASAC and should result in a more effective statement to the Administrator 
regarding the science that should be used as a factor in making policy judgments. 

Sincerely, 

James D. Crapo, M.D. 
Professor of Medicine 
National Jewish Medical and Research Center 

and 
Director, Clinical Science Ph.D. Program 
University of Colorado Health Sciences Center 

National Jewish Medical and Research Center 

1400 Jackson Street, Denver, CO 80206 

Tel: 303-398-1436; Fax: 303-270-2243; E-mail: crapoi@nic.ore 

http ://w w w . National Jewi sh . or g 



Case 1:05-cv-01814-LFO Document 23-5 Filed 05/31/2006 Page 53 of 77 

Attachment 3-D 

Dr. Frederick J. MUler 



Comments on the NAAQS process 

Frederick J. Miller 

March 24, 2006 

Over the last 30 years, I have seen the NAAQS process unfold from both sides of the 
table - first as an EPA employee coauthoring chapters in Criteria Documents (CD) and 
interacting with members of CASAC, later as an ad hoc CASAC reviewer, and since 
2000 as an EPA appointed regular member of CASAC. Others who have served on 
CASAC have provided their insights and suggestions for improving the NAAQS process. 
I offer the following comments on the strengths and weaknesses of the NAAQS process 
and how, in my opinion, this process needs to be changed. 

Timeliness and Efficiency of the Current NAAQS Review Process 

With each review cycle for a NAAQS pollutant, the number of available scientific 
publications has grown exponentially. Because the Agency has viewed the CD as needing 
to be exhaustive in the review of all available peer-reviewed papers, the NAAQS CDs 
have greatly expanded in size, thereby requiring longer and longer periods of time for 
review and more iterations if the documents are not of high quality or if they contain 
biased discussions of the studies. This has led to court ordered time schedules and a less 
than desirable process - in short, the current process is broken and needs to be fixed. 

Recommendation - Change the structure and focus of the CD 
We need to know if effects are occurring below the levels of current NAAQS standards 
and if current standards are adequately protective of public health. Thus, the main focus 
of the CD should be to identify and discuss any new studies, for which there will usually 
be few, that establish effects below the levels of current standards . In addition studies that 
are relevant to the indicator for the standard, the averaging time, and the statistical form 
should be included. All other studies should be relegated to appendices tables if the 
Agency is required to or wants to be "complete" in its review of the literature. 

If the structure and focus of the CD were done as suggested above, CASAC and Agency 
staff could have more interactive discussions on these results and interpretations of these 
critical studies. CASAC would be in a better position to identify if there are other studies 
that are relevant to the indicator variable, averaging time, numerical level, or statistical 
form. And most importantly, discussions at CASAC meetings could focus on whether or 
not current standards protect public health with an adequate margin of safety. This new 
focus might also enable the CD and Staff Paper to be combined into a single document, 
as has been recommended by CASAC member. Dr. James Crapo. 

Consideration of the Most Recent Available Science 

Since scientific research is iterative, there is no magical date when all is known about an 
issue. The lengthy time for preparation and review of the CD and Staff Paper for a 
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NAAQS pollutant is a by-product of the current process. If the CD structure and focus is 
changed as suggested above, the ability to incorporate the most recent available science 
would be greatly improved because the time interval for the process would be 
significantly shortened compared to what it is now. 

Recommendation - Incorporate critical new science 
If the CD focuses on studies that impact our knowledge about pollutant effects at or 
below the levels of current standards, there should be ample time in the review cycle to 
consider critical new science. The new science must, however, meet the following 
criteria: (1) be judged to be of such a nature that it could change the indicator variable, 
averaging time, numerical level, or statistical form of the standard, and (2) have been 
reviewed and vetted by CAS AC. The second criterion is absolutely essential for 
maintaining the objectivity, credibility, and integrity of CAS AC in fulfilling its statutory 
mandate. I agree with others who have noted that the recent EPA management change 
eliminating the long-standing finalization of the CD and the Staff Paper via a CAS AC 
closure letter was unwise. The inclusion of newly available science should not be done by 
bypassing CASAC review of this new science. 
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Additional CASAC Member Individual Comments on EPA's NAAQS Process 



Dr. Sverre Vedal 
March 29, 2006 

I would like to comment briefly on two of the key questions posed to CASAC panelists. 

Consideration of the most recent available science 

First of all, I am concerned that the science that we have available for review already is not 
providing us with a complete, or entirely valid, picture. This contention is based on the well- 
documented publication bias present in the air pollution field, at least for population-based 
findings. An illuminating demonstration of this bias is the contrast between conclusions 
contained in Health Effects Institute (HEI) scientific reports and some published journal articles 
derived from the same findings. While there are undoubtedly several causes of this publication 
bias, investigators, journal editors, and even the journal peer review process, are partly to blame. 
What is the alternative to limiting what we consider science for the purpose of the CASAC 
review to the published literature? While I am not intending to promote HEI specifically, one 
precedent was recently set by HEI in addressing the statistical software problem in time series 
studies. This process occurred outside the traditional publication process and provided EPA with 
the information needed in an efficient and credible manner. Generalizing such a process to the 
task of assessing larger scientific questions may hold some promise and be worthy of 
consideration. Clearly this would take resources that are currently not available. 

Identifying, characterizing, quantifying, and communicating uncertainties in scientific 
information 

The uncertainties in the epidemiological estimates of effect, estimates that are now the primary 
bases for recommendations on changes to the NAAQS, are not adequately reflected in the 
current risk assessment and risk analyses. A formal probabilistic risk assessment that fully takes 
into account identified uncertainties is needed. 

Sverre 



Sverre Vedal, MD 

Department of Environmental and Occupational Health Sciences 

University of Washington 

4225 Roosevelt Way NE, Suite 100 

Seattle, WA 98105-6099 

phone: 206-616-8285 

fax: 206-685-4696 

email: svedal@u.washineton.edu 
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Attachment 4-A 
Summary of Major Stakeholder Comments^ 



Planning 

Criteria for selecting studies 

• A careful set of criteria for studies should be defined a priori and reviewed by CAS AC 

o Could support meta-analysis (N) 
Identify issues with management 

• At the beginning of the process, identify issues that need to be focused on in the next review, 
and get senior (political) management input at that time as to whether that is the direction 
where they want us to go (I) 

Maintain status quo 

• Generally, thinks the process works very well and is not "broken"; applauds OAQPS (S) 

• Does not want to see any major changes (E) 
Resources 

• Wonders whether resources are adequate (S) 

• On the timeliness issue, very simple answer: EPA needs to devote adequate (well-allocated) 
resources to make it happen every 5 years (E) 

Welfare effects 

• Don't forget about welfare effects (E) 



Science assessment 

Time between "closure" and final regulation 

• Time between closure and proposal an issue (N) 

o Perhaps the concern about the "newest" science is overstated - very rare that a single 
new study (or a small handful) is that important 

• Remember, new science is not always the best science. Need defensible criteria to determine 
best science (S) 

o Having clear criteria could help CASAC in its review 

Continuous study collection with criteria for evaluation 

• EPA needs to take advantage of electronic methods better to collect information better on 
studies (N) 

o Current reliance on big tables and long text is cumbersome. Possible models: 

■ Database used in the Surgeon General's report 

■ NAS PM Research committee database 

■ Anderson's work for the European Union 

■ HEI's work in Asia 



Note: N = National Academy of Sciences (NAS) chairs 
I = Industry stakeholders 
S = State stakeholders 
E = Environmental stakeholders 
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o A Microsoft Access database, set up correctly, can be used in a lot of different ways, 
and could serve more than the needs of the NAAQS (N) 

■ Need young computer whizzes on staff to do this - get people in-house 

■ Developing such a database should be continuous and not tied to the 
development of a specific Criteria Document (N) 

• The Agency should have an ongoing process for reviewing new studies as they appear. The 
Criteria Document could just compile these reviews, with description of uncertainties (I) 

• Early on, do a notice requesting papers to submit, set clear criteria up front for how EPA 
would review these studies. Bring all the uncertainties forward in the write-up, but note that 
this does not mean ignore if there is uncertainty. (I) 

• Continual data collection (surveying scientific literature) rather than doing everything all at 
once, using a database (E) 

• Need clear guidance on how studies are to be reviewed (S) 

• EPA needs a continuous review process (S) 

o Perhaps EPA could do an annual report on new science (at least for PM and ozone) 
and get CASAC review to answer whether the new science changes the overall view 
of the science - giving a head-start on the Criteria Document process 

Need for a more concise science assessment/criteria document 

• Current model for synthesis (200-page "synthesis" with a paragraph at the end that pulls it all 
together) is not the best approach (N) 

• Criteria Document should be a true update rather than a rehash of older work (N) 

• Criteria Document is too long (S) 

• Combine Criteria Document and Staff Paper into a single step with one peer review (S) 

o Others on this call believe these should be maintained as distinct documents; there is 
already some overlap in their development, but perhaps this could be increased 

• Need to explicitly compare what we knew in the last review vs. what we know now (I) 

Uncertainty 

• Criteria Document needs to be better in its qualitative approach to describing uncertainty (N) 

• There are different types of uncertainty - real scientific uncertainty and people who just 
disagree with the science outcome. When Criteria Document cites "other points of view," 
need to be explicit about whose views these are to assess whether honest scientific arguments 
or something else (E) 

• Stakeholders always want more analysis. Need to discuss strength/weakness of studies and 
try to understand uncertainties for standard setting, but can go overboard (S) 



Risk/exposure assessment 

Positive statements re: risk assessment 

• Risk assessments done by OAQPS are quite thorough and transparent (N) 

• Sensitivity analyses that are done now are very helpful and well done (E) 

Recommended improvements 

• Expedite implementation of the NAS recommendations on benefits assessment (I) 

• The choices made in the assessment need to be clearly delineated. Needs to be more 
transparency in the assessments to highlight what we look at or choose not to look at (I) 
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Negative statement 

• "Risk" is just a means of "writing off some anonymous individuals who will be hurt or 
killed by air pollution (E) 



Policy assessment 

Avoid premature review 

• First draft of Staff Paper often brought to CAS AC in a form that is too incomplete (N) 

Eliminate Stajf Paper 

• Get rid of the Staff Paper and put all that information into an ANPRM for everyone 
(including CAS AC) to review just once (I) 

Maintain current approach 

• EPA risk assessment and Staff Paper do an excellent job getting at uncertainties; CASAC is 
the cause of EPA having to chase down a lot of insignificant loose ends (S) 

• Staff Paper is a good thing - should not be done or influenced by political leadership. Keep 
politics out of it (E) 



CASAC review and advice on criteria and standards 

Need clearer guidance as to CASAC role 

• CASAC needs clear guidance — that it can speak of the adequacy of standards, but that its 
arguments must be grounded in science (N) 

o Need to better articulate CAS AC 's scientific rationale for judgments; population 
protected (a policy call) versus the nature of the dose response curve (science) 

• CASAC should be reviewing what management wants to do, not staff recommendations 

o But if we want this, need to add policy specialists on CASAC 

• Need a "lecture" at the beginning of the process for CASAC members — that they advise but 
do not make policy (I) 

Need for formal "closure" 

• Get rid of "closure" (S) 

Bias 

• Need to address the issue of bias (real or perceived). For example, with respect to CASAC 
members (I) 

o the choice of CASAC members; 

o concern that certain members are reluctant to contradict EPA staff because that might 

stop the flow of grand money; 
o lead reviewers of individual chapters who also wrote cited papers; 
o publication bias (always mentioned, then swept under rug); 
o insufficient time for public comment at meetings 
o inappropriate contact between EPA staff and CASAC members (e.g., CASAC 

dinners) 
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o inappropriate conduct of CASAC members during public comments (e.g., side 

conversations) 
o members who have clear policy goals in mind 

How to address CASAC comments 

• EPA needs to do a better job of managing CASAC process. Need to hear from all members, 
but perhaps not necessary to address every individual comment (S) 

o EPA is too deferential 

o Perhaps use decision science to identify what the important issues/uncertainties are 
and limit resources on other topics, even if CASAC members are stressing them 

Issues related to CASAC roster, structure 

• Perhaps there should be a CASAC subcommittee that reviews the risk assessment (N) 

• Would be a good idea to have a State technical and policy person (2 individuals) on CASAC 
(S) 

o It should be science only - keep policy people off CASAC (differing state opinions) 



Rulemaking 

Decouple regulatory process from scientific assessment process 

• The short time for review is a problem for EPA; concern that decisions must be made 
before science is fully vetted. Perhaps when the clock runs out, EPA could as a matter of 
routine, just reaffirm and let the process run its course (I) 

Continual management role 

• Maintain management communication throughout (I) 

Decision criteria 

• Science/policy distinctions? A fallacy. NAAQS must be set on science, protect health 
with adequate margin of safety. NAAQS decisions should not be based on other 
considerations such as how nonattainment is affected (E) 

• Uncertainty - must err on the side of protecting public health, whole reason for an 
adequate margin of safety (E) 

Concerns regarding current process review 



A post- April 3 process is needed before any recommendations are acted upon (E) 

Several NESCAUM member States expressed concern about their lack of participation 

(S) 

Consider input from states other than NESCAUM and California (I) 
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JOHNS HOPKINS Attachment 4 B 

UNIVERSITY 



Bloomberg School of Public Health 

615 North Wolfe Street / Suite W6041 

Baltimore, MD 21205-2179 

(410) 955-3286 / FAX (410) 614-0467 

E-mail: isamet@ihsph.edu 

www.ihsph.edu/epi 

Department of Epidemiology 

Office of the Chairman 



March 13,2006 

Memorandum 

TO: Lydia Wegman, Kevin Teichman 

FROM: JonSamet \ . "S^^^^ 

SUBJECT: NAAQS Process 

I am writing to elaborate on comments made during our telephone discussion on Monday, March 6*. I 
am pleased to have the opportunity to provide input as the US Environmental Protection Agency (EPA) 
considers the process by which the National Ambient Air Quality Standards (NAAQS) are set. This 
process now has long historical precedent and it has proven successful in many respects. However, as I 
noted in our discussion, the process has become "dated" and could be made more efficient and 
informative. 

Any revisions to the process should reflect the increasing challenge posed by the extent of the scientific 
information available in regard to the setting of a particular NAAQS . In the example of particulate matter 
(PM) the peer-reviewed literature available, even for a five-year interval, is extraordinary in its diversity 
and abundance. The long-used format for gathering the evidence for assessment, assembling a large, 
tabular volume, is no longer effective. In the instance of epidemiologic research alone, tables may 
include numerous studies, easily 100 or more in some lines of investigation, and characterizing the key 
advances made from bullet tables and tedious text is very difficult. 

Consequently, I suggest consideration of moving to a database approach that would better use available 
information management technology. As an example, we have developed an Access database that has 
been used for assembling evidence on smoking and health. Core information about each publication is 
complemented by tables that capture the study data. Once all of the information is abstracted into the 
database, there is potential to generate files for analysis, including quantitative summaries using meta- 
analysis. The project carried out by Ross Anderson and colleagues at St. George's in London is another 
model; Ross has been supported by WHO-Euro to assemble a database of the findings of time-series 
studies. The time-series study example is useful, as many such studies have been published, and any 
synthesis of the evidence needs to explore heterogeneity and to look at summary measures. 

The EPA also needs to give formal consideration to methods for evidence synthesis. For example, what 
is the role of quantitative meta-analysis? Should Bayesian approaches be considered? 

The current NAAQS process has a mandated five-year time frame, although that period is almost 
invariably exceeded. The development of the Criteria Document usually begins several years in 
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advance of anticipated deadline and often moves hastily at its end. As an alternative, I suggest that EPA 
maintain a unit with the dedicated task of keeping an up-to-date database of evidence. Done properly, 
this database could serve the information needs for Criteria Documents on the various Criteria pollutants; 
in fact, the same study may be relevant. 

Another major issue for consideration is the process for moving from the Criteria Document to the Staff 
Paper and its recommendations. The algorithms for this translational step are varied and, in the recent 
example of PM, have included a risk assessment. The conduct of a risk assessment was informative in 
this instance, but the policy framework for interpreting the results was not explicit. The process would be 
better served if the translational steps were better spelled out so that there was greater transparency. The 
review of the Staff Paper by CAS AC might be more effective if the underlying framework were evident. 
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Attachment 4-C 

The Clean Air Association of the Northeast States 



101 Merrimac Street, 1 0th Floor Boston, MA 02 1 1 4 

Phone 61 7-259-2000 Fax 61 7-742-9162 

Arthur N.Marin, Executive Director 
www.nescaum.org 

March 15,2006 

Lydia Wegman 

Office of Air Quality Planning and Standards 
U.S. Environmental Protection Agency 
Research Triangle Park, NC 2771 1 

Re. Key Questions for the Review of the Process for Setting NAAQS 

Dear Ms. Wegman: 

NESCAUM offers the following comments and suggestions regarding the review of EPA's 
process for setting National Ambient Air Quality Standards (NAAQS) based on observations 
stemming from our participation in the recent particulate matter NAAQS review. NESCAUM is 
the regional association of air pollution control agencies representing Connecticut, Maine, 
Massachusetts New Hampshire, New Jersey, New York, Rhode Island, and Vermont. 

Overall, the process for setting NAAQS works reasonably well at the EPA staff and the Clean 
Air Scientific Advisory Committee (CASAC) level, except for the habitual tardiness of 
completing the review. Agency staff and CASAC are to be commended for their comprehensive 
and dedicated efforts. NESCAUM offers the following perspective regarding the key questions 
to which you have solicited feedback. These thoughts are offered in addition to the feedback 
provided on our March 3, 2006 telephone conversation with EPA staff and the State of 
California. 

Timeliness of the NAAQS review process 

An overriding concern is the overly long time between the start of the review and the publication 
of the Agency's proposed decisions on standards. EPA should strictly follow the Clean Air 
Act's requirement to conduct reviews every five years. Given the difficulty of achieving this 
requirement, management efforts should focus anew on how best to assure this objective. Its 
successful implementation will minimize unnecessary and preventable public health impacts. 
Possible suggestions - some of which may already be in place - include: 

• Conduct the Criteria Document literature review early on as a cumulative multi-year 
process with formal report summaries every six to 12 months. These could be internal 
documents or shared externally. Consider this process as creating preparatory interim 
materials for the first draft of the Criteria Document (CD). The expectation would be to 
avoid a massive effort to evaluate years of material simultaneously; invariably such 
"crash" efforts take longer than expected and overwhelm staff. If EPA already uses such 
a process, consider expanding its scope so that each interim deliverable accomplishes as 
much as possible (e.g., literature review, analysis, interpretation, integration) as soon as 
possible (e.g., within the first 1-3 years of the next review cycle). Each subsequent report 



NESCAUM Members: Massachusetts Bureau of Waste Prevention, Barbara Kwetz New York Division of Air Resources, David Siiaw 

Connecticut Bureau of Air Management, Anne Gobin New Hampshire Air Resources Division, Robert Scott Rhode Island Office of Air Resources, Stephen Majl<ut 
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• 



• 



• 



could integrate previous materials so that the final interim report will represent a 
cumulative approximation of a first draft CD. This could also help reduce CASAC's 
work load in reviewing early CD drafts. 

Likely this process would require pre-established transparent protocol regarding various 
criteria, including literature selection and analysis procedures. It would also require 
organizational cohesion and suppleness across EPA subject-area groups. Preferably, this 
protocol would provide staff with clear guidelines so that each interim report was subject 
to the same "recipe" of steps. This could help to ensure consistency and continuity 
within each review process as well as across separate review processes. It could also 
facilitate understanding among CASAC and public participants of analytical methods 
EPA uses when preparing the CD. 

Reduce the length of the CD. View the document as an update of science rather than an 
encyclopedic report. To the extent EPA can conduct timely 5 -year reviews, this will help 
to reduce the tendency of CDs to become unwieldy. 

Evaluate the feasibility of concurrently preparing the CD and Staff Paper (SP). Revisit 
original assumptions and arguments made that advocated for the temporal separation of 
each document's preparation. If the cumulative CD process described above is used (or 
augmented, if already used), perhaps early iterations of the SP could be prepared based 
on early interim CD deliverable reports. 

EPA's relation to the CASAC deliberation process should continue to maintain Agency 
deference accorded to individual and collective committee views and proceedings. 
CASAC's internal deliberation process, however, should incorporate additional 
mechanisms to limit the influence that certain members could exert if this influence 
unduly slows or roils the review process. Do not allow actions to exert a pell-mell and 
instantaneous effect on the committee's schedule and overall decision-making process. 
Otherwise, throughout the course of 2-3 years such delays can snowball, placing a 
difficult and perhaps unnecessary burden on other CASAC members and EPA staff. 
Perhaps CASAC leaders could communicate more definition and expectation to 
committee members before the process begins; perhaps employ more "rules of order" 
during committee proceedings. I recall examples where a very small CASAC contingent 
appeared to invoke a final hour public comment or overly dwell on scientific uncertainty 
with the effect of delaying both CASAC and EPA's ability to move forward with the 
revision process. The collective effect of these instances was likely nontrivial. It might 
be worth analyzing CASAC meeting transcripts to identify the frequency and severity of 
such cases. While it is commendable to act upon public comments and raise uncertainty 
issues, some type of leadership oversight, group consensus, and vigilant awareness 
should accompany these actions. This would ensure minimization of inconsequential 
concerns that could negatively impact process timeliness and schedule-keeping. 
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• It is important that CASAC and EPA continue to provide adequate opportunities for 
public comment. The periodic scheduling approach of reducing individual comments 
from 5 minutes to 3 minutes should be reconsidered. For commenters that have devoted 
extensive resources and time to preparing detailed materials that make use of original 
data, 3 and even 5 minutes can be insufficient. Nonetheless, the wisdom of providing 
equal time to all parties should prevail. The recent practice of allowing groups to stack 
individual public comment periods covering separate topics seems arbitrary and requires 
the group to expend more resources to attend the meeting. This could give an unfair 
advantage to groups with substantial resources. Regardless of what means EPA uses to 
schedule public comments, the Agency should provide an explicit policy well in advance 
of the meeting rather than right before or during the meeting, whenever possible. 

• Given their heavy workload and responsibilities in reviewing EPA materials, is it realistic 
to assume that CASAC members will have adequately reviewed public comments 
materials submitted to them in advance? NESCAUM experienced this effect: CASAC 
members who had not devoted time to deliberation of public comment materials received 
in advance of the meeting admitted they had "not been aware of materials from our 3- 
minute presentation. Afterwards, they wished they had been aware of these materials 
during meeting deliberations. Perhaps EPA could prepare a document that overviews 
public comment materials that are submitted in advance. The Agency could share this 
overview document with CASAC members. This might make it easier for CASAC 
members to consider public comments, especially those comments that provide data 
analysis deemed by EPA as relevant and credible. 

Consideration of the most recent available science 

• If EPA can adhere to a strict 5 -year schedule then the Agency will not table (for upwards 
of a decade) studies that appear at the end of the review cycle and which miss cutoff 
dates. The above comments on timeliness of review using a cumulative yearly process 
could serve to shorten the time between the presumptive cutoff date for scientific studies 
evaluated in the review and proposed decision dates. 



• 



EPA's Administrator should not base standard-setting decisions on studies that CASAC 
has not reviewed and which have not been subject to public comment. Such a last-minute 
process could undermine the peer review and public accessibility intent of the Clean Air 
Act. The management problem leading to lack of timely NAAQS revision should not 
interfere with the need for transparency and orderly scientific review. If EPA considers 
studies published after the cutoff date so ground-breaking and important that they require 
inclusion, then EPA could conduct a mini-review subject to CASAC and public 
comments. Regardless, the Agency should discourage such a process and lay out in 
advance criteria for deciding that a late-breaking study is ground-breaking. 
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• The Agency should lay out criteria used to determine what science is "best" and to 
delineate what science is "most recent." 

• Is the "best" and "most recent" science necessarily the strongest science? Care should be 
taken in determining how these concepts are understood and utilized. The "newness" of 
studies should not be construed as a priori more relevant or stronger than "older" studies 
in a manner that mistakenly accentuates their bearing on the process. The potential for 
this is likely more subconscious than conscious, and should be guarded against. "Best" 
science may not be the most relevant science, depending on how it is defined. For 
example, an innovative and nimble small-scale study with plausible findings based on 
new data might not be as rigorous as a large-scale comprehensive study with airtight 
findings replicating previous studies. But the public health relevance of the former study 
could be more important. What if emerging hypothesis-driven data capture an important 
public health concern that should be addressed precisely because the consequences of not 
acting could outweigh the consequences of waiting? Should the process detach these 
potentially informative findings from decision makers? This could have the unintended 
result of relegating important information into lesser categories of "non-best" science. 
Does the current process ignore that data until it becomes absolute, if this is how "best" 
science is defined? Is the "best" science good enough to identify public health risks? 
Perhaps plausibility should be incorporated into any criteria used to define "best" and 
"most recent" available science. Moreover, lack of timeliness in conducting NAAQS 
reviews every five years would exacerbate the potential negative impact of these 
considerations. 

• Is the "best" and "most recent" science actually "available" in a comprehensive way? 
Academic frameworks of knowledge pursuit largely drive the current composition and 
manner of CASAC. These frameworks often collect knowledge generated from within 
their sphere of familiarity. Local, state, and federal frameworks using knowledge 
generated from within their respective spheres within the arena of public health decision 
making are responsible for the implementation of standard setting. How certain are we 
that these spheres overlap? To what extent do NAAQS reviews incorporate local and 
state public health science? To what extent is academic science incorporated into 
everyday local and state decision making? Such considerations could recommend state 
public health agency representation on CASAC as well as within the formulation of 
criteria for "best" and "most recent" document selection used in the decision-making 
process for setting NAAQS. 

• Broad multi- and interdisciplinary staff expertise - both within EPA and across to 
relevant public health agencies (local, state, federal, tribal) with overlapping interventions 
and mandates relating to ambient air quality - should be emphasized when considering 
"best" and "most recent" and "available" science. Periodic assessments of this capacity 
should occur. 
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Distinctions between science and policy judgments 

• Science should inform policy judgments; policy should follow the science. Science 
should not be misconstrued or unduly influenced to support policy goals based on issues 
other than health. If policy judgments are expected to be based upon scientifically- 
informed public health decision making, then they should be, with no exceptions. 

Identifying, characterizing, quantifying, and communicating uncertainties in scientific 
information 

• Oyer time, scientific uncertainty will be resolyed incrementally. The more current and 
frequent the chance to update science to inform standard setting, the better public health 
protection society will enjoy. From a public health perspectiye, decision making with 
absolute certainty is unrealistic. Public health decision making must be conducted in 
spite of uncertainty. 



• 



Uncertainty should not be construed as an excuse to delay decision making or to 
minimize public health concerns stemming from decision making. Nor should 
uncertainty be construed as a pejoratiye concept. Uncertainty is a part of the scientific 
process. Thus, the 5 -year reyiew cycle should be adhered to strictly as science adds to 
our knowledge oyer time. Timely NAAQS reyiew is perhaps the best remedy for 
addressing scientific uncertainty in standard setting. 

• CASAC should determine how best to handle the numerous uncertainties of scientific 
information they are tasked to reyiew. The committee should not use uncertainty as an 
excuse for delay or fixation. Rather, when issues of uncertainty arise - whether small, 
moderate, or large - the committee should coUectiyely haye a clear-cut method of 
addressing uncertainty and then moye on toward completing its job in a timely manner, 
uncertainty notwithstanding. 



Thank you for the opportunity to proyide these perspectiyes. 
Sincerely, 



Philip Johnson 
Senior Scientist 



cc: NESCAUM Directors 

Arthur Marin, Executiye Director 
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AMERICAN LUNG ASSOCIATION 
ENVIRONMENTAL DEFENSE 

March 22, 2006 

Mr. Marcus Peacock 

Deputy Administrator 

U.S. Environmental Protection Agency 

1200 Pennsylvania Ave. N.W. 

Washington, D.C. 20460 

Re: Requested Comments on the Process for Setting NAAQS 

Dear Deputy Administrator Peacock, 

This letter is in response to your staffs invitation to comment on the process for setting National 
Ambient Air Quality Standards, as part of a review you requested in a December 15, 2005 
memo. 

On January 18, 2006, we wrote you the attached letter expressing our concern that the process 
outlined in your memo was far too abbreviated to provide for adequate public input and review. 
We expressed our concern that any reconsideration of the process for setting the NAAQS must 
allow for a full and open opportunity for public comment. To date you have not responded to our 
letter. 

We therefore respectfully reiterate our concerns about the approach EPA is undertaking. First 
and foremost, to the extent that a review is necessary, EPA must ensure that the process is 
transparent and open to full input from both the expert scientific community and the lay public. 
The extremely limited amount of time that you have set for this process hampers meaningful 
public involvement. Indeed, we ask you to re-consider this accelerated, truncated and opaque 
process and to instead engage in a deliberative, transparent and inclusive dialogue with the wide 
range of affected interests. The Agency has provided scant public information about the 
concerns that have prompted this review. At the same time, only an extremely limited set of 
organizations and individuals are even aware that this review is taking place. The inaccessible 
rationale for this process and the limited opportunity for informed public involvement have 
precluded the meaningful public input that is the hallmark of rigorous public policy. 

The NAAQS review process must meet two requirements mandated in the Clean Air Act: that 
the review of the science and the standard occur every five years and that the result is a standard 
that protects public health with an adequate margin of safety. These dual pillars comprise the 
statutory foundation for evaluating the NAAQS process. 

The extant NAAQS review process succeeds in three essential respects. First, the process 
provides frequent and significant opportunities for public comment, including opportunities to 
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comment on the work plan for the review, protocols, the draft criteria document, staff paper and 
risk assessment, and on CASAC's letters and input. Second, the criteria documents developed 
for each NAAQS review provide valuable and thorough reviews of the science of air quality 
impacts on public health. Third, the process allows independence for EPA staff scientists to 
develop recommendations that are purely based on the science as "requisite to protect the public 
health" and provide "an adequate margin of safety." 

Prompt and regular reviews are required by law and are essential to protect human health. 

If EPA seeks to improve the process, we recommend allocating sufficient staff and resources to 
complete the reviews on the required five-year schedule. Congress clearly intended that the 
public benefit from the most current scientific understanding of the health risks from air 
pollution. EPA's failure to meet those cycles is not just a missed deadline; failure means that 
some of our most vulnerable citizens literally risk life and health because dangerous levels of air 
pollution are not recognized and addressed promptly. Furthermore, keeping to the five-year 
cycles resolves the issue of how to incorporate emerging research. With a regular review cycle, 
unlike the current pattern, emerging research becomes grist for the analysis that will immediately 
follow. 

In addition to sufficient resources, we would recommend the following measures to shorten the 
time required within each review period: 

• EPA should examine whether use of automated search processes, publication 
notification services, improved database technology and electronic publishing could 
streamline and modernize the literature search and assessment process by enabling a 
continuous review of new scientific literature as it is published. 

• NCEA staff should summarize the conclusions in the last criteria document and focus 
their efforts on reviewing research published since the last review, rather than 
beginning anew with the science in each cycle. The criteria document should clearly 
identify concentrations at which effects are observed. 

• OAQPS staff should use the assessment of the science from the criteria document in 
the staff paper, and concentrate on original analysis of air quality, health impacts, and 
sensitive populations and other analyses that inform the policy recommendations. 

The result must protect public health with an adequate margin of safety. 

The second requirement is to produce a standard that protects public health with an adequate 
margin of safety. The basis of the standard must be a scientific review of objective peer- 
reviewed research and analysis. 

The Act requires an "independent, scientific review committee" to review the science and to 
recommend any revisions to the standard. The clear direction of this was to create an open, 
transparent process for scientific review and deliberation to provide an independent assessment 
of the science. Without question, this open review is integral to this process and must not be 
short-circuited. The CASAC review process has consistently provided open public discussion 
and assessment. 
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The staff paper likewise provides valuable scientific recommendations from the staff to the 
Administrator, with the opportunity to provide such recommendations insulated from political 
and economic considerations, consistent with the Clean Air Act. Like the criteria document, the 
staff paper is subject to peer review and thoroughly vetted in open forum. The process must 
allow the EPA staff to develop recommendations that are exclusively based on the public health 
science, without consideration of either politics or cost. This valuable process has stood the test 
of time. 

As the reviewing courts have repeatedly held, the Clean Air Act clearly separates the process of 
setting NAAQS from the process of implementing them. If the Administrator follows that 
dictate, the decision to set the NAAQS therefore will be based on science — on what limits should 
be set to protect public health with a margin of safety. Policy considerations such as 
implementation priorities and costs come later during the implementation of those standards. 

Fundamental to the protection of public health is the requirement to include the margin of safety. 
From the beginning Congress intended this to be the key means to address the issues of 
uncertainty. Even during the drafting of the Clean Air Act, the writers understood clearly that 
uncertainty would always exist and developed the margin of safety requirement to adapt to that 
reality. Analytical efforts to quantify uncertainties in scientific information must not delay public 
health protection. 

We hope that our comments are useful in clarifying ways that will both promote the review of 
the science and the standards every five years and produce standards that protect public health 
with an adequate margin of safety. We also reiterate our request for the Agency to re-consider 
the current process. We respectfully ask you to instead carry out a process that is inclusive, 
transparent and informed by a more rigorous dialogue with the interested public and clearer 
illumination of the Agency's own rationale for undertaking this process. 

Sincerely, 



Paul G. Billings 

Vice President, National Policy and Advocacy 

American Lung Association 



Jana Milford 
Staff Scientist 
Environmental Defense 



Cc: George Gray, Ph.D., Assistant Administrator for Research and Development 
William Werhum, Assistant Administrator for Air and Radiation 
Lydia Wegman, OAQPS 
John Bachmann, OAQPS 
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National Environmental Trust * National Parks Conservation Association * Natural Resources Defense 

Council * Physicians for Social Responsibility * Sierra Club * Union of Concerned Scientists 

U.S. Public Interest Research Group 

January 18,2006 

Mr. Marcus Peacock 

Deputy Administrator 

U.S. Environmental Protection Agency 

1200 Pennsylvania Ave. N.W. 

Washington, D.C. 20460 

Dear Deputy Administrator Peacock, 

We have reviewed your memorandum of December 15, 2005 establishing an EPA 
working group to conduct a "top to bottom" review of the National Ambient Air Quality 
(NAAQS) review and standard setting process. While we welcome all appropriate efforts 
to strengthen the NAAQS review and setting process, we are concerned about the 
potentially political nature of this review. We strongly encourage you to ensure that any 
changes to the NAAQS process be considered and adopted using a process that allows the 
same kind of scientific and public input afforded by the standard-setting process itself 

The stated purpose of the review is to ensure that the NAAQS process adheres to the 
highest scientific standards. The memorandum requests a broad review that would 
examine elements of the NAAQS process that are mandated by the Clean Air Act as well 
as elements that are not. The review will include both the preparation of the Criteria 
Document and the review and setting of NAAQS standards which are separate mandates 
under the Act. 

As you know, the establishment of the National Ambient Air Quality Standards is the 
foundation of the Clean Air Act from which crucial activities under the Act designed to 
reduce air pollution and protect public health and the environment derive. The process 
EPA uses to prepare and adopt Criteria Documents and review and establish NAAQS 
standards has evolved over many years in response to statutory mandate, scientific 
method, and regulatory and administrative custom. This process provides a highly 
refined evaluation of the latest scientific knowledge to obtain measures needed to protect 
public health and the environment while accommodating broad scientific and public 
input. The current process, which is the fundamentally same as was used to establish the 
ozone and PM 2.5 NAAQS in 1997, has withstood substantial judicial scrutiny. (See 
American Trucking Assn. v. USEPA, 175 F.3d 1055-56 (D.C. Cir. 1999)) 

Your memorandum sets an April 3 deadline for recommended changes. We believe this 
is far too short a period to solicit, obtain and evaluate recommended changes to the 
NAAQS process from the broad spectrum of stakeholders who participate in the standard 
setting process. 
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Second, we believe recommendations should be sought from the Clean Air Scientific 
Advisory Committee as well as the Clean Air Act Advisory Committee both of which 
have significant expertise on the establishment and implementation of NAAQS standards. 
Consultation with the Children's Health Protection Advisory Committee would also be 
appropriate, given the impact of air pollution on children and babies. 

Finally, to the extent EPA identifies changes to the NAAQS review and standard setting 
process that it endorses, it should seek public comment on such changes prior to their 
adoption. In particular, EPA should seek public comment on any changes it supports that 
require amendments to the Clean Air Act. 

We share your goal of ensuring that the best available science guide and inform EPA 
decision making in the NAAQS review and standard setting process. We believe that the 
current process does this well. We urge a careful and open process for identifying 
NAAQS process improvements and look forward to the opportunity to providing our best 
recommendations and advice. 

Sincerely, 



Paul G. Billings 

Vice President, 

National Policy & Advocacy 

American Lung Association 



Karen Way land 

Legislative Director 

Natural Resources Defense Council 



Frank O'Donnell 
Executive Director 
Clean Air Watch 

Conrad G Schneider 
Advocacy Director 
Clean Air Task Force 

Elizabeth Thompson 
Legislative Director 
Environmental Defense 

John Stanton 

Vice President 

National Environmental Trust 

Mark Wenzler 

Vice President, 

Legislative Affairs 

National Parks Conservation Association 



Kyle Kinner 
Acting Director, 
Policy and Programs 

Environmental Health 
Physicians for Social Responsibility 

Debbie Sease 
Legislative Director 
Sierra Club 

Alden Meyer 

Director of Strategy and Policy 

Union of Concerned Scientists 

Anna Aurelio 

Legislative Director 

U.S. Public Interest Research Group 



Cc: Stephen Johnson 
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A ^^ .^,„ 1220 L Street, Northwest Kyle B. Isakower 

American Washington, DC 20005-4070 Director, Policy Analysis 

Petroleum 202-682-8314 

Tnetihito 202-682-8330 Fax 

insumie isakowerk@api.org 



March 27, 2006 

Ms. Lydia Wegman 

Director, Air Quality Strategies and Standards Division 

USEPA, Office of Air Quality Planning and Standards 

Mail Code: C504-02 

Research Triangle Park, NC 2771 1 

Dr. Kevin Teichman 

Director, Office of Science Policy 

USEPA, Office of Research and Development 

Ariel Rios Building 

1200 Pennsylvania Avenue, N. W. 

Mail Code: 8104R 

Washington, DC 20460 USEPA 



Re: API Comments on the National Ambient Air Quality Standards Setting 
Process Review 



This letter constitutes the comments of the American Petroleum Institute (API) on EPA's 
current review of the National Ambient Air Quality Standards (NAAQS) standard-setting 
process. API is the primary trade association of America's oil and natural gas industry, 
and represents more than 400 members involved in all aspects of the oil and natural gas 
industry. It draws on the experience and expertise of its members and staff to support a 
strong and viable oil and natural gas industry. API members are dedicated to continuous 
efforts to improve the compatibility of their operations with the environment, while 
economically developing energy resources and supplying high quality products and 
services to consumers. 

As the owners and operators of the most of the nation's refineries, and as the suppliers of 
much of the Nation's energy, API's members have a significant stake in the effective and 
efficient development of the NAAQS under the Clean Air Act (CAA). We appreciate the 
Environmental Protection Agency's (EPA or Agency) fact finding efforts to suggest 
improvements to the periodic review process for those standards. API historically has 
played an active role throughout the NAAQS review process, including reviewing and 
commenting on the drafts of Criteria Documents (CD) and Staff Papers (SP) for various 
criteria pollutants. API has also provided comments to the Clean Air Science Advisory 



An equal opportunity employer 
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Committee (CASAC) when it has met to review NAAQS-related documents and develop 
recommendations to the Administrator. 

API was pleased to participate in a stakeholder call held by the Agency on February 23, 
2006, and submits these written comments to augment the discussion. 

1 . The Staff Paper is ^policy document, and as such, should have input from 
senior EPA management. 

Although the Clean Air Act requires that NAAQS be established based on science and 
without regard to cost, EPA's assumptions of "no threshold" and linear exposure 
response relationships essentially make establishment of any non-zero risk standard a 
policy decision, not a scientific one. It is therefore inappropriate and unreasonable to have 
these policy decisions debated by career staff, CASAC scientists and the media without 
any input from senior EPA policymakers. 

The Act has no requirement for a SP, nor does it include any prohibition against EPA 
senior management participating in the process. Either senior management should 
participate early, and throughout, the SP process, or the SP should be eliminated 
altogether, replaced by an Advanced Notice of Proposed Rulemaking (ANPRM) and 
stakeholder forums to provide additional input. 

Alternatively (or perhaps in addition), policy makers should be added as consultants to 
the CASAC panel, to better educate the technical panel members in policy-related issues 
surrounding standard setting. 

2. The 5-year review of NAAQS is insufficient to adequately review a NAAQS; 
the requirement for review should be modified legislatively to an "as needed" 
basis. 

Although the current review is targeted at administrative improvements, raising the need 
for a statutory change to the 5-year review cycle is warranted in the context of improving 
the NAAQS development process. The current process for revising a standard is a tedious 
and time-consuming one that involves a lengthy Agency assessment of the relevant health 
research and development of a CD which is then reviewed by CASAC. This often 
requires multiple drafts and reviews prior to fmalization. EPA staff then develops a SP, 
also reviewed by CASAC, which contains their standard-setting recommendations. This, 
too, often requires muUiple drafts and reviews. EPA then generates a proposed rule to 
modify or reaffirm the existing standards, along with an accompanying Regulatory 
Impact Analysis, and other risk analyses documents. Only after the process is complete 
can the Administrator, in conjunction with staff and interagency review, develop a final 
rule. This process clearly requires more than 5 years to complete. 
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API recommends a legislative change to require EPA to modify criteria pollutant 
NAAQS only "as needed," so that a thorough review of the science and policy issues 
surrounding a NAAQS review can be conducted without the artificial and unreasonable 
time constraints currently imposed by the CAA. Further, from a practical standpoint, 
there should be no new NAAQS established until the current NAAQS has been 
implemented. 

Alternatively, to protect against overly extended periods between reviews, such 
legislation could include a requirement that a NAAQS review for each criteria pollutant 
be initiated within 5 years of promulgation of the previous standard or 1 year of final 
designations of non-attainment with the previous standard, whichever is later. 

This approach would also reduce the potential for future litigation forcing EPA into 
settlements with individual stakeholders. EPA should not be forced to enter into consent 
decrees that mandate timetables to review NAAQS and to complete related rulemakings 
(e.g., implementation) without the opportunity for broad stakeholder input. 

Detractors from the above approach may claim that environmental quality could suffer; 
API would disagree. Due to implementation of the existing NAAQS through State 
Implementation Plans (SIPs), as well as the Clean Air Interstate Rule (CAIR), Tier 2 
mobile source requirements, oflf-and on-road vehicle rules mandating ulfra-low sulfur 
diesel fuels and accompanying vehicular controls. New Source Performance Standards 
(NSPS), Prevention of Significant Deterioration (PSD) and New Source Review (NSR) 
requirements and Maximum Achievable Control Technology (MACT) rules, extending 
the 5-year review to "as needed" would not reverse current emission reduction trends. 
While additional research is pursued, the nation's air quality will continue to improve, 
and public health protection will continue to increase due to the programs already in 
place. 

3. Until legislation can address the 5-year review requirement through 

legislation, EPA should simply reaffirm the existing standard if the review is 
not complete within 5 years. 

This reaffirmation approach has been used in the past (most recently by Administrator 
Carol Browner) where EPA indicated that the evaluation of the science did not warrant a 
modification to the NAAQS at that time. It did not - nor would it in the future - mean 
that the review would have to wait an additional 5 years. If the review required, say, 7 
years, then reaffirmation could occur 5-years after the previous final rule, with another 
final rule promulgated 2 years after reaffirmation, starting the 5-year clock again. 
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4. The Agency should review new NAAQS-related research as it is made 
available, with liey information placed in a publicly available database. 

This would provide a less rushed and more complete review process. The logging of 
these studies into a public database would better ensure all studies are given equal weight 
and add transparency to the review of scientific information. 

5. The SP would be strengthened with a formal inclusion of the disbenefits 
concept, as mandated by the courts. 

American Trucking Associations, Inc., etal. v. EPA, 175 F. 3d 1027 (D.C. Cir., 1999), 
required that EPA include a formal evaluation of the disbenefits of proposed rules. 
Examples of these "missing" disbenefits include: 

a. The "NOx disbenefit.' NOx control hastens formation of marginally 
reduced peak ozone in the more populated areas, thereby increasing net 
population exposure and effects; 

b. The 'UVB disbenefit.' Ozone control permits higher surface level UVB 
exposure thereby increasing population skin cancer, cataracts, and immune 
deficiency; 

c. A potential 'VOC mineralization disbenefit.' Ozone control reduces 
atmospheric OH radical levels and consequently the rate of VOC photo 
oxidation, thereby increasing atmospheric lifetimes of organic air toxics. 

Similarly, EPA should recognize and evaluate the broader public health implications of 
imposing costly standards that offer limited and uncertain health benefits. There is an 
apparent disbenefit that the Agency has not addressed to date, regarding the linkages 
between regulatory costs and family health. Serious health problems arise when a 
family's living standards decline. Costly government regulations adversely affect 
productivity, which in turn dampens real income. There is an income-mortality 
connection that needs to be addressed. Greater attention to such risk comparisons could 
resuh in valuable gains to the environment and public health. 

To date, EPA has not incorporated an adequate review of these disbenefit issues into their 
NAAQS review process, despite Court mandates to do so. Greater effort must be put on 
evaluating disbenefits to ensure the adequacy of future NAAQS reviews. 
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6. The opportunity for bias or lack of candor must be removed from the 
CASAC review process. 

As discussed on the February 23 conference call, there are numerous instances where at 
least the opportunity for, or the appearance of bias, is present in the CASAC review 
process. The following areas of concern were specifically identified: 

a. There is no transparency in the selection or retention of CASAC members 
or consultants; 

b. Some CASAC members/consultants who disagree with EPA conclusions 
may be reluctant to stand up for their positions, given that some 
members/consultants may be seeking fiiture research grants from EPA, or 
may be concerned that their research may be reviewed by colleagues on 
CASAC (who tend to agree with EPA) in the fiiture; 

c. Some CASAC members may be reluctant to insist on changes, given the 
short timefi-ame and EPA's "we must meet our deadline" stance; 

d. CASAC's lead reviewers for specific chapters in CD or SP reviews are 
often the very same researchers who conducted the studies discussed in 
that chapter. This clearly introduces the potential for an appearance of bias 
in the review of the chapters; 

e. Publication bias is often discussed in CASAC meetings, but never taken 
into account in conclusions. Publication bias, the tendency [on average] to 
produce results that appear significant, because negative or near neutral 
results are almost never published, can be statistically significant. If it is 
accepted that a CASAC review will almost certainly be subject to 
publication bias to some extent, that leaves the problem of estimating how 
big a problem it is, and what to do about it; 

f. Experts whose opinions vary fi-om EPA's - and are not chosen to be 
incorporated into EPA's conclusions - are not given equal time to present 
their findings to CASAC; 

g. Public comments often do not appear to be seriously considered within the 
CASAC process. The time allotted to the public at CASAC meetings is 
often insufficient to adequately present oral testimony. In addition, some 
CASAC members often appear distracted by other activities/conversations 
during the public testimony period; and 
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h. There has been contact between EPA staff and CASAC (private meeting 
rooms, social dinners, etc.) that could give EPA staff at least the 
appearance of additional opportunities to convince CASAC members of 
their own positions. 

While some of the issues listed above may, in fact, be perceptual problems, rather than 
examples of true bias, EPA must nonetheless address these issues directly. Only by doing 
so can stakeholders have confidence in the adequacy and objectivity of NAAQS reviews. 

7. EPA must fully respond to public comment througbout the review process. 

The public is encouraged to provide written comments to EPA on draft CD and SP 
documents. However, EPA often fails to address these comments when developing their 
next draft, instead focusing only on issues raised by CASAC. This becomes even more 
problematic when EPA establishes a deadline for public comment that follows the 
CASAC meeting. In such cases, not only does EPA often not address public comments, 
but CASAC cannot even review them. While EPA will address many public comments in 
the proposed rule, it is well after the issues have been debated by CASAC and staff 
opinions have been presented in the media. 

API appreciates that the Agency is reviewing the current standard-setting method to both 
improve the evaluation of the science, and incorporate policy makers throughout the 
process. API suggests the EPA revisit the efficacy of the current process and modify it to 
allow a measured review of evolving knowledge and experience. Part of that review may 
lead the Agency toward a method used in the past with a new standard that was not ripe; 
namely, reaffirming the existing standard. 

Please feel fi-ee to contact me with any questions regarding these comments. I look 
forward to your final report. 



Sincerely, 




Kyle B. Isakower, Director 
Policy Analysis 



